-- Logs begin at Thu 2022-09-08 15:27:50 UTC, end at Thu 2022-09-08 16:24:24 UTC. -- Sep 08 15:53:19.280652 dv-cinder-856259-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 08 15:53:19.288289 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 08 15:53:19.290340 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 8 15:53:19 2022] *** Sep 08 15:53:19.290340 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 08 15:53:19.290340 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 08 15:53:19.290340 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: nodename: dv-cinder-856259-2-iscsi Sep 08 15:53:19.290340 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: machine: x86_64 Sep 08 15:53:19.290340 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: clock source: unix Sep 08 15:53:19.290340 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: pcre jit disabled Sep 08 15:53:19.290340 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: detected number of CPU cores: 4 Sep 08 15:53:19.290340 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: current working directory: / Sep 08 15:53:19.290340 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: detected binary path: /usr/bin/uwsgi-core Sep 08 15:53:19.290340 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: your processes number limit is 47680 Sep 08 15:53:19.290340 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: your memory page size is 4096 bytes Sep 08 15:53:19.290340 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: detected max file descriptor number: 2048 Sep 08 15:53:19.290340 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: lock engine: pthread robust mutexes Sep 08 15:53:19.291183 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: thunder lock: enabled Sep 08 15:53:19.291183 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 08 15:53:19.291183 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 08 15:53:19.337517 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: Python main interpreter initialized at 0x5578e12879c0 Sep 08 15:53:19.337517 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: python threads support enabled Sep 08 15:53:19.337517 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: your server socket listen backlog is limited to 100 connections Sep 08 15:53:19.337517 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: your mercy for graceful operations on workers is 90 seconds Sep 08 15:53:19.337928 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: mapped 403077 bytes (393 KB) for 2 cores Sep 08 15:53:19.337995 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: *** Operational MODE: preforking *** Sep 08 15:53:19.338578 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: *** uWSGI is running in multiple interpreter mode *** Sep 08 15:53:19.338578 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: spawned uWSGI master process (pid: 81514) Sep 08 15:53:19.338816 dv-cinder-856259-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 08 15:53:19.339899 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: spawned uWSGI worker 1 (pid: 81526, cores: 1) Sep 08 15:53:19.339899 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: spawned uWSGI worker 2 (pid: 81527, cores: 1) Sep 08 15:53:19.340050 dv-cinder-856259-2-iscsi devstack@keystone.service[81514]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 08 15:53:23.582096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 15:53:23.582096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: from cryptography.utils import int_from_bytes Sep 08 15:53:23.582907 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 15:53:23.582907 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: from cryptography.utils import int_from_bytes Sep 08 15:53:23.583320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 15:53:23.583320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: from cryptography.utils import int_from_bytes Sep 08 15:53:23.583422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 15:53:23.583422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: from cryptography.utils import int_from_bytes Sep 08 15:53:23.632728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 15:53:23.632916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 15:53:23.633006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 15:53:23.633084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 15:53:23.633145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] command line args: [] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 15:53:23.633202 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] command line args: [] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 15:53:23.633361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 15:53:23.633491 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 15:53:23.633568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ================================================================================ {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 15:53:23.633657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ================================================================================ {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 15:53:23.633730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] admin_token = **** {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.633880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] admin_token = **** {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.634024 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] config_dir = [] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.634100 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] config_dir = [] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.634158 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.634280 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.634365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] config_source = [] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.634533 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] config_source = [] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.634668 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] debug = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.634751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] debug = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.634819 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.635048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.635145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.635212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.635271 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.635333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.635519 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.635599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.635672 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.635738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.635943 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] list_limit = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.636068 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] list_limit = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.636140 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] log_config_append = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.636199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] log_config_append = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.636269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.636335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.636398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] log_dir = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.636457 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] log_dir = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.636610 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] log_file = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.636688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] log_file = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.636797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.636874 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.637035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.637116 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.637296 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.637418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.637505 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.637570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.637714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.637816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.637881 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.637955 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.638088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.638172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.638355 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.638486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.638577 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.638654 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.638774 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.638851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.639035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.639159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.639241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.639327 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.639481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.639617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.639813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.640003 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.640003 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.640287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.640523 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.640656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.640746 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] publish_errors = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.640827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] publish_errors = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.640899 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.641028 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.641104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.641183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.641309 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.641390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.641557 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.641709 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.641709 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.641859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.641968 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] standard_threads = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.642060 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] standard_threads = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.642213 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.642349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.642430 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.642508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.642587 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.642719 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.642796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] use_journal = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.642935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] use_journal = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.642935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] use_json = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.643067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] use_json = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.643142 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] use_stderr = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.643213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] use_stderr = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.643288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] use_syslog = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.643413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] use_syslog = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.643513 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.643596 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 15:53:23.643746 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.643868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.643955 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.644037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.644217 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.644400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.644476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.644551 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.644742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.644832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.644977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.645058 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.645223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.645304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.645455 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.645543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.645699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.645788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.645942 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.646028 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.646195 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.646287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.646427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.646507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.646672 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.646752 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.646904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.646995 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.647146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.647231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.647390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.647509 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.647645 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.647733 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.647897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.647983 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.648148 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.648228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.648381 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.648454 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.648612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.648704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.648864 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.648943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.649106 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.649182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.649357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.649431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.649603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.649683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.649833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.649914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.650062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.650136 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.650293 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.650368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.650528 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.650604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.650773 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.650847 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.651017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.651097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.651246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.651323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.651494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.651573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.651742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.651832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.651987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.652064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.652222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.652296 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.652469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.652543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.652701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.652784 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.652936 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.653031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.653172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.653287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.653404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.653546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.653673 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.653800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.653928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.654048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.654253 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.654380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.654500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.654620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.654754 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.654882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.654882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.655121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.655247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.655366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.655499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.655616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.655745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.655868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.655988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.656106 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.656224 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.656449 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.656593 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.656725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.656866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.656991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.657122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.657251 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.657386 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.657471 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.657585 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.657693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.657779 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.657863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.657939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.658053 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.658133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.658204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.658315 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.658400 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.658477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.658548 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.658659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.658745 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.658828 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.658909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.658981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.659111 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.659223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.659297 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.659407 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.659535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.659608 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.659694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.659768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.659841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.659962 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.660044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.660268 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.660392 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.660477 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.660644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.660766 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.660957 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.661079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.661170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.661246 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.661425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.661548 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.661621 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.661883 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.662017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.662099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.662272 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.662394 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.662581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.662713 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.662804 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.662958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.663080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.663258 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.663389 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.663493 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.663694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.663817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.663982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.664114 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.664196 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.664347 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.664483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.664662 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.664785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.664900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.664987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.665161 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.665336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.665430 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.665619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.665741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.665820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.666069 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.666277 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.666367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.666441 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.666512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.666679 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.666816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.666903 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.666984 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.667145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.667268 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.667347 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.667419 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.667629 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.667765 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.667846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.667918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.668087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.668211 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.668288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.668375 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.668536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.668661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.668739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.668811 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] auth.external = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.668984 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.669123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.669210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.669283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.669444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.669568 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.669682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.669766 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] auth.password = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.669905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.669997 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] auth.token = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.670157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.670240 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.670401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.670483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.670639 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.670729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.670866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.670956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] auth.external = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.671094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.671180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.671333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.671418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.671589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.671681 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.671830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] credential.caching = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.671940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] auth.password = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.672056 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] auth.token = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.672147 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.672293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.672379 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.672527 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.672606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.672774 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.672861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.673002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.673079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.673238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.673315 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.673478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.673562 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.673707 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.673785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.673941 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] credential.caching = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.674018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.674186 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.674275 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.674435 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.674512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.674684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.674761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.674925 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.675025 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.675169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.675247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.675403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.675511 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.675725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] federation.caching = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.675817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.675964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.676041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.676209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.676286 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.676455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.676540 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.676684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.676762 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.676930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.677008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.677165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.677244 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.677417 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.677495 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.677651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.677728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.677904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.677982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] federation.caching = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.678137 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.678219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.678381 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.678457 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.678615 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity.caching = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.678694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.678844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.678920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.679089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.679166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.679323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.679401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.679580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.679675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.679820 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.679896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.680048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.680126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.680270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.680347 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.680516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.680594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.680754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity.caching = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.680831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.680988 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.681064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.681219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.681295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.681449 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.681528 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.681678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.681754 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.681922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.682005 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.682151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.682230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.682384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.682461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.682631 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.682708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.682881 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.682960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.683116 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.683265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.683386 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.683531 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.683627 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.683703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.683818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.683896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.684052 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.684135 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.684291 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.684370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.684535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.684614 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.684780 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.684856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.685027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.685104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.685259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.685343 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.685499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.685576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.685733 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.685812 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.685969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.686249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.686249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.686389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.686470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.686543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.686650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.686756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.686891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.687007 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.687183 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.687289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.687513 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.687599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.687712 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.687805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.687961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.688041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.688197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.688273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.688444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.688573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.688692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.688831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.688909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.689086 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.689163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.689322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.689402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.689556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.689693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.689808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.689887 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.690051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.690131 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.690289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.690367 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.690528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.690673 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.690759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.690930 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.691008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.691213 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.691291 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.691465 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.691560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.691703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.691781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.691956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.692034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.692206 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.692292 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.692444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.692523 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.692680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.692756 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.692911 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.692997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.693142 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.693218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.693374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.693451 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.693611 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.693699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.693843 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.693921 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.694090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.694167 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.694336 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.694421 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.694580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.694657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.694816 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.694894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.695047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.695133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.695283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.695359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.695554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.695697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.695795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.695965 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.696044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.696226 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.696304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.696475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.696551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.696721 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.696800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.696966 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.697042 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.697200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.697455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.697539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.697687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.697766 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.697930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.698106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.698184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.698255 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.698390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.698473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.698624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.698702 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.698859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.698946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.699098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.699176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.699329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.699407 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.699594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.699694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.699829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.699906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] resource.caching = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.700076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.700153 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.700308 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.700394 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.700549 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.700626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.700782 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.700859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.701016 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.701102 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.701251 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.701330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.701497 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.701574 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.701732 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.701816 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.701960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.702081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.702194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] role.caching = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.702272 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.702432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] resource.caching = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.702510 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] role.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.702662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.702739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.702910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.702988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.703143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.703218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.703380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.703479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.703653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.703731 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.703888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.704212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.704355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.704591 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.704824 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] role.caching = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.705069 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] role.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.705383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.705634 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.705711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.705884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.705962 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.706134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.706221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.706383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.706461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.706625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.706702 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.706922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.707013 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.707169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.707248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.707416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.707557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.707671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.707764 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.707921 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.708128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.708207 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.708398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.708643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.709077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.709294 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.709374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.709546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.709623 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.709795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.709881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.710045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.710122 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.710296 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.710375 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.710532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.710612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.710786 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.710863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.711035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.711113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.711273 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.711353 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.711530 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.711611 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.711762 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.711839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.711992 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.712072 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.712242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.712326 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.712494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.712573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.712744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.712823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.713113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.713549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.713775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.714008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] token.caching = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.714243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.714490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.714725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.714982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.715230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.715499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.715826 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.715906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.716163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.716287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.716411 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.716661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] token.caching = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.716747 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.716910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.717066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.717172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.717315 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.717420 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.717558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.717689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.717808 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.717947 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.718092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.718195 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.718325 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.718445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.718568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.718695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.718835 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.718948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.719099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.connection = **** {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.719185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.719344 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.719423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.719620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.719702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.719860 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.719955 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.720106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.720184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.720340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.720419 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.720576 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.720663 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.720812 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.721211 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.721297 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.721511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.721753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.721987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.722233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.722493 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.722737 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.722966 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.723205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.723437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.723626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81527) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 15:53:23.724192 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.724482 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.connection = **** {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.724740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.724997 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.725242 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.725489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.725728 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.725967 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.726205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.726443 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.726689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.726927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.727174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.727411 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.727690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.727930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.728177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.728412 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.728656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.728895 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 15:53:23.729067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81526) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 15:53:23.753087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.753354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.753554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.757661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.757896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.758124 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.761979 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.766456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.789822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.790064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.792321 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.792561 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.843709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.844244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.891032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.891268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.892564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:23.892802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.050396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.050608 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.050855 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.051073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.051290 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.051483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.054607 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.054847 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.055078 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.055299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.055541 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.055765 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.077365 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.077538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.077623 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.077757 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.077870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.077948 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.078020 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.078203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.078339 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.078425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.078508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.078612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.078703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.078881 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.090281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.090551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.103173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.103304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.103616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.103813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.103909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.104037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.104161 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.104282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.104403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.104571 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.104694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81526) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.104821 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.104821 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.105071 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81527) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 15:53:24.121534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.121666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.121820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 15:53:24.121947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 15:53:24.122070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.122191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.122367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 15:53:24.122459 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 15:53:24.122583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.122663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.122794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.122886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.123007 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.123166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.123287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.123421 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.132864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.133012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.133119 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.133210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.133424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.133555 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.133673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.133766 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.140636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.140763 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.143489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.143631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.143809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.143941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.144093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.144218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.144364 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.144462 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.144667 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.144793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.144907 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.144999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.145197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.145326 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.145471 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.145598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.145738 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.145877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.145987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.146079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.146261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.146387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.168686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.168813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.170301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.170416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.170416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.170535 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.175044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.175172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.175325 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.175469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.175653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.175798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.181924 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.182045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.182183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.182267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.187926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.188056 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.188231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.188355 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.188507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.188652 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.188764 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.188856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.189063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.189189 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.189342 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.189479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.189589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.189681 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.189850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.189943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.205420 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.205546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.205694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.205823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.209585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.209782 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.212811 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.212939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.213088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.213592 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.217615 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.217742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.220737 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.220865 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.224054 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.224339 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.224466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.224620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.224746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.224870 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.225005 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.225136 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.225260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.225413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.225542 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.225729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.241146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.241379 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.241479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.241681 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.241805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.241908 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.241987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.242197 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.248748 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.249010 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.250296 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.250562 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.251840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.252106 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.252233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.252383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.254875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.255097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.255199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.255388 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.255501 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.255702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.255792 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.255974 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.256118 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.256398 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.267075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.267353 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.267514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.267717 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.267847 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.268053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.268148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.268271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.268392 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.268572 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.268673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.268866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.268995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.269122 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.284008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.284363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.286936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.287306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.289832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.290222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.291332 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.291645 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.292825 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.293104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.293229 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.293488 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.293673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.293883 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.299826 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.300190 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.301295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.301675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.304453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.304580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.304847 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.304994 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.305141 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.305361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.305455 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.305583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.313105 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.313233 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.313455 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.313557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.313690 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.313814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.320565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81527) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.320768 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81526) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 15:53:24.320874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.321026 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.321149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.321270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.321390 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.321518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.321647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.321768 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.321925 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.322048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.322183 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.322331 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.322466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.322593 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.322729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.322855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.322982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.323108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.323234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.323404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.323556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.323680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.323834 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.323963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.324086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81527) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.324219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81526) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 15:53:24.358212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5578e12879c0 pid: 81526 (default app) Sep 08 15:53:24.359130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5578e12879c0 pid: 81527 (default app) Sep 08 15:53:24.361579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8ed95-f891-4fc8-8cac-d0a46325a0ad None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:24.361866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8ed95-f891-4fc8-8cac-d0a46325a0ad None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:24.362142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8ed95-f891-4fc8-8cac-d0a46325a0ad None None] PATH_INFO: `/v3` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:24.366420 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Thu Sep 8 15:53:24 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:31.073271 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea0cf5-8b1b-49a5-b7de-45968046717d None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:31.073820 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea0cf5-8b1b-49a5-b7de-45968046717d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:31.073820 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea0cf5-8b1b-49a5-b7de-45968046717d None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:31.080829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:31 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:31.087047 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6936a-0624-4753-8bf1-b6f110c19908 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:31.087749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6936a-0624-4753-8bf1-b6f110c19908 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:31.087749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a6936a-0624-4753-8bf1-b6f110c19908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:31.157480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: INFO dbcounter [None req-84a6936a-0624-4753-8bf1-b6f110c19908 None None] Registered counter for database keystone Sep 08 15:53:31.178570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG oslo_db.sqlalchemy.engines [None req-84a6936a-0624-4753-8bf1-b6f110c19908 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81526) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 15:53:31.186600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writer thread running {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 15:53:31.370149 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG passlib.handlers.bcrypt [None req-84a6936a-0624-4753-8bf1-b6f110c19908 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81526) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 08 15:53:31.370439 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG passlib.handlers.bcrypt [None req-84a6936a-0624-4753-8bf1-b6f110c19908 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81526) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 08 15:53:31.428874 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-84a6936a-0624-4753-8bf1-b6f110c19908 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:31.430479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-84a6936a-0624-4753-8bf1-b6f110c19908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:31.564787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 08 15:53:32.033495 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea0cf5-8b1b-49a5-b7de-45968046717d None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:32.105387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: INFO dbcounter [None req-40ea0cf5-8b1b-49a5-b7de-45968046717d None None] Registered counter for database keystone Sep 08 15:53:32.130006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG oslo_db.sqlalchemy.engines [None req-40ea0cf5-8b1b-49a5-b7de-45968046717d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81527) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 15:53:32.138184 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writer thread running {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 15:53:32.256040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea0cf5-8b1b-49a5-b7de-45968046717d None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:32.261516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c5349-55d0-4ad4-919a-cff6f879fb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:32.262149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c5349-55d0-4ad4-919a-cff6f879fb90 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:32.262425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c5349-55d0-4ad4-919a-cff6f879fb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:32.262690 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c5349-55d0-4ad4-919a-cff6f879fb90 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:32.271726 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c5349-55d0-4ad4-919a-cff6f879fb90 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:32.326463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ae7c5349-55d0-4ad4-919a-cff6f879fb90 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 08 15:53:32.326735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c5349-55d0-4ad4-919a-cff6f879fb90 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:32.336990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Thu Sep 8 15:53:32 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 15:53:32.347217 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6936a-0624-4753-8bf1-b6f110c19908 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:32.371398 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a6936a-0624-4753-8bf1-b6f110c19908 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:32.377088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89dad9c-410c-43aa-9ae8-a70ad55d5fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:32.378233 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dad9c-410c-43aa-9ae8-a70ad55d5fdf admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:32.378517 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dad9c-410c-43aa-9ae8-a70ad55d5fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:32.378788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89dad9c-410c-43aa-9ae8-a70ad55d5fdf admin admin] PATH_INFO: `/v3/projects/e21b84c9f53a40aa98d4a318db545714` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:32.380477 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89dad9c-410c-43aa-9ae8-a70ad55d5fdf admin admin] RBAC: Authorizing `identity:get_project(project_id=e21b84c9f53a40aa98d4a318db545714)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:32.432081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e89dad9c-410c-43aa-9ae8-a70ad55d5fdf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 08 15:53:32.432425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89dad9c-410c-43aa-9ae8-a70ad55d5fdf admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:32.436923 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Thu Sep 8 15:53:32 2022] GET /identity/v3/projects/e21b84c9f53a40aa98d4a318db545714 => generated 344 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:34.550428 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a5335-f501-4168-8b22-15ceae4f395d None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:34.551068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a5335-f501-4168-8b22-15ceae4f395d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:34.551068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405a5335-f501-4168-8b22-15ceae4f395d None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:34.555227 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:34.562446 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fd992-c02a-4f57-b1d1-7be1df1302be None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:34.562856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fd992-c02a-4f57-b1d1-7be1df1302be None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:34.563007 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fd992-c02a-4f57-b1d1-7be1df1302be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:34.591293 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-953fd992-c02a-4f57-b1d1-7be1df1302be None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:34.592831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-953fd992-c02a-4f57-b1d1-7be1df1302be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:34.602269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 08 15:53:34.872065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a5335-f501-4168-8b22-15ceae4f395d None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:34.895739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405a5335-f501-4168-8b22-15ceae4f395d None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:34.901043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9993cd-d427-48ea-91d7-885926ff4570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:34.901653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9993cd-d427-48ea-91d7-885926ff4570 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:34.901931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9993cd-d427-48ea-91d7-885926ff4570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:34.902199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9993cd-d427-48ea-91d7-885926ff4570 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:34.910334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9993cd-d427-48ea-91d7-885926ff4570 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:34.912172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9993cd-d427-48ea-91d7-885926ff4570 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:34.920877 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Thu Sep 8 15:53:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 15:53:34.926721 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fd992-c02a-4f57-b1d1-7be1df1302be None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:34.936042 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fd992-c02a-4f57-b1d1-7be1df1302be None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:34.942228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a71331-a960-42ff-bffe-ec6d669853bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:34.942863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a71331-a960-42ff-bffe-ec6d669853bc admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:34.943135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a71331-a960-42ff-bffe-ec6d669853bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:34.943399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a71331-a960-42ff-bffe-ec6d669853bc admin admin] PATH_INFO: `/v3/users/02fc092547d04cbf984e632c0755920a` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:34.944891 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a71331-a960-42ff-bffe-ec6d669853bc admin admin] RBAC: Authorizing `identity:get_user(user_id=02fc092547d04cbf984e632c0755920a)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:34.946728 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a71331-a960-42ff-bffe-ec6d669853bc admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:34.948289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Thu Sep 8 15:53:34 2022] GET /identity/v3/users/02fc092547d04cbf984e632c0755920a => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:37.150218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f86790-e7fd-476a-93f5-525999536bf3 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:37.150797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f86790-e7fd-476a-93f5-525999536bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:37.150797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f86790-e7fd-476a-93f5-525999536bf3 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:37.152127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:53:37.157621 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b1abe-9dd4-4f12-829f-36a99f1e3376 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:37.158029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b1abe-9dd4-4f12-829f-36a99f1e3376 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:37.158265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b1abe-9dd4-4f12-829f-36a99f1e3376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:37.162438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a17d2-fe9f-4ad7-98ea-2c01c841ddd3 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:37.162682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a17d2-fe9f-4ad7-98ea-2c01c841ddd3 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:37.162980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a17d2-fe9f-4ad7-98ea-2c01c841ddd3 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:37.164209 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 6/11] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:53:37.170040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08345c21-13d1-4855-bd49-773f3ca24fc5 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:37.170320 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08345c21-13d1-4855-bd49-773f3ca24fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:37.170645 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08345c21-13d1-4855-bd49-773f3ca24fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:37.181630 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-121b1abe-9dd4-4f12-829f-36a99f1e3376 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:37.182936 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-121b1abe-9dd4-4f12-829f-36a99f1e3376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:37.215923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG passlib.handlers.bcrypt [None req-08345c21-13d1-4855-bd49-773f3ca24fc5 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81527) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 08 15:53:37.216312 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG passlib.handlers.bcrypt [None req-08345c21-13d1-4855-bd49-773f3ca24fc5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81527) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 08 15:53:37.264595 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-08345c21-13d1-4855-bd49-773f3ca24fc5 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:37.266151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-08345c21-13d1-4855-bd49-773f3ca24fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:37.267861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 6/12] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:37.367073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:37.533815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b1abe-9dd4-4f12-829f-36a99f1e3376 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:37.560150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:37.638107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08345c21-13d1-4855-bd49-773f3ca24fc5 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:37.646971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b1abe-9dd4-4f12-829f-36a99f1e3376 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:37.680791 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a6f3b-fee2-41a5-8d11-d12639e18fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:37.681430 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a6f3b-fee2-41a5-8d11-d12639e18fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:37.681701 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a6f3b-fee2-41a5-8d11-d12639e18fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:37.681967 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a6f3b-fee2-41a5-8d11-d12639e18fa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:37.693103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a6f3b-fee2-41a5-8d11-d12639e18fa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-796a6f3b-fee2-41a5-8d11-d12639e18fa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:53:37.701964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:53:37.702892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:53:37.703793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:53:37.703793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:53:37.703793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:53:37.703793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:53:37.703793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:53:37.703793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:53:37.703793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:53:37.703793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:53:37.703793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:53:37.703793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:53:37.703793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:53:37.703793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:53:37.703793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 7/14] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:53:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:53:37.709132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a6f3b-fee2-41a5-8d11-d12639e18fa8 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:37.741463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08345c21-13d1-4855-bd49-773f3ca24fc5 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:37.744736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:37.779917 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbe721-914e-41eb-a8d7-3dc8a7a96ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:37.780800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbe721-914e-41eb-a8d7-3dc8a7a96ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:37.781078 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbe721-914e-41eb-a8d7-3dc8a7a96ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:37.781348 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cbe721-914e-41eb-a8d7-3dc8a7a96ae5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:37.789748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:37.809141 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a6f3b-fee2-41a5-8d11-d12639e18fa8 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:37.843340 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224945c3-ed6a-4b81-87e5-eb906557155a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:37.843982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224945c3-ed6a-4b81-87e5-eb906557155a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:37.844255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224945c3-ed6a-4b81-87e5-eb906557155a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:37.844523 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224945c3-ed6a-4b81-87e5-eb906557155a None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:37.845207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224945c3-ed6a-4b81-87e5-eb906557155a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:37.845508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224945c3-ed6a-4b81-87e5-eb906557155a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:37.846919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224945c3-ed6a-4b81-87e5-eb906557155a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:37.849516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbe721-914e-41eb-a8d7-3dc8a7a96ae5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:37.851275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cbe721-914e-41eb-a8d7-3dc8a7a96ae5 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:37.862351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 8/15] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:53:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:53:37.868097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224945c3-ed6a-4b81-87e5-eb906557155a None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:37.947783 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 8/16] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:53:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:53:37.953429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbe721-914e-41eb-a8d7-3dc8a7a96ae5 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:37.956596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:37.968897 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224945c3-ed6a-4b81-87e5-eb906557155a None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:38.002171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275571c-0f1e-4961-884a-3a7d30a84bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:38.002801 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275571c-0f1e-4961-884a-3a7d30a84bbe None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:38.003073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275571c-0f1e-4961-884a-3a7d30a84bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:38.003340 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275571c-0f1e-4961-884a-3a7d30a84bbe None admin] PATH_INFO: `/v3/users/02fc092547d04cbf984e632c0755920a` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:38.004537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275571c-0f1e-4961-884a-3a7d30a84bbe None admin] RBAC: Authorizing `identity:get_user(user_id=02fc092547d04cbf984e632c0755920a)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:38.006007 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275571c-0f1e-4961-884a-3a7d30a84bbe None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:38.007646 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 9/17] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:53:37 2022] GET /identity/v3/users/02fc092547d04cbf984e632c0755920a => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:53:38.013157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275571c-0f1e-4961-884a-3a7d30a84bbe None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:38.027184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:38.054451 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cbe721-914e-41eb-a8d7-3dc8a7a96ae5 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:38.087581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42227bd-ed36-42b9-981a-6e8692cc9d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:38.088897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42227bd-ed36-42b9-981a-6e8692cc9d18 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:38.089171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42227bd-ed36-42b9-981a-6e8692cc9d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:38.089438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42227bd-ed36-42b9-981a-6e8692cc9d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:38.108194 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42227bd-ed36-42b9-981a-6e8692cc9d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:38.109686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42227bd-ed36-42b9-981a-6e8692cc9d18 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:38.113398 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275571c-0f1e-4961-884a-3a7d30a84bbe None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-f42227bd-ed36-42b9-981a-6e8692cc9d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:53:38.128022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:53:38.129147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:53:38.130023 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:53:38.130023 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:53:38.130023 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:53:38.130023 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 9/18] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:53:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:53:38.135004 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42227bd-ed36-42b9-981a-6e8692cc9d18 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:38.152715 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86906356-88b1-4461-808a-ca45ec1f9125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:38.153361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86906356-88b1-4461-808a-ca45ec1f9125 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:38.153692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86906356-88b1-4461-808a-ca45ec1f9125 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:38.153967 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86906356-88b1-4461-808a-ca45ec1f9125 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:38.155086 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86906356-88b1-4461-808a-ca45ec1f9125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:38.156585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86906356-88b1-4461-808a-ca45ec1f9125 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:38.158146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 10/19] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:53:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:53:38.163978 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86906356-88b1-4461-808a-ca45ec1f9125 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:38.195182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:38.242325 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42227bd-ed36-42b9-981a-6e8692cc9d18 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:38.242526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:38.265270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86906356-88b1-4461-808a-ca45ec1f9125 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:38.275994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725464ed-f2aa-4b07-ad6b-1cf367c5b589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:38.276608 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725464ed-f2aa-4b07-ad6b-1cf367c5b589 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:38.276882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725464ed-f2aa-4b07-ad6b-1cf367c5b589 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:38.277147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725464ed-f2aa-4b07-ad6b-1cf367c5b589 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:38.277791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725464ed-f2aa-4b07-ad6b-1cf367c5b589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:38.278089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725464ed-f2aa-4b07-ad6b-1cf367c5b589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:38.279498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725464ed-f2aa-4b07-ad6b-1cf367c5b589 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:38.296290 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 10/20] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:38.299954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d3ba47-f71c-4999-9aad-237212bf1c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:38.300643 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3ba47-f71c-4999-9aad-237212bf1c17 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:38.300899 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3ba47-f71c-4999-9aad-237212bf1c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:38.301170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d3ba47-f71c-4999-9aad-237212bf1c17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:38.301910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3ba47-f71c-4999-9aad-237212bf1c17 None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.domain.id=default, user.id=02fc092547d04cbf984e632c0755920a) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:38.302211 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3ba47-f71c-4999-9aad-237212bf1c17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:38.303718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d3ba47-f71c-4999-9aad-237212bf1c17 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:38.311858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 11/21] 10.222.0.27 () {56 vars in 1322 bytes} [Thu Sep 8 15:53:38 2022] GET /identity/v3/role_assignments?user.id=02fc092547d04cbf984e632c0755920a&scope.domain.id=default&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:53:40.766079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ae400-e9d2-442c-8fc0-bb7d5eee7166 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:40.767467 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ae400-e9d2-442c-8fc0-bb7d5eee7166 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:40.767972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ae400-e9d2-442c-8fc0-bb7d5eee7166 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:40.769863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 11/22] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:40.774937 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099c105-4619-4a23-83a8-686fa684c3ca None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:40.775773 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099c105-4619-4a23-83a8-686fa684c3ca None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:40.776351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9099c105-4619-4a23-83a8-686fa684c3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:40.798961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-9099c105-4619-4a23-83a8-686fa684c3ca None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:40.800559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-9099c105-4619-4a23-83a8-686fa684c3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:40.869331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec7df1-89e5-4013-aa34-cd7d507f5fd3 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:40.869976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec7df1-89e5-4013-aa34-cd7d507f5fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:40.870443 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec7df1-89e5-4013-aa34-cd7d507f5fd3 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:40.872293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 12/23] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:40.880787 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea7755-1328-4642-8d53-cfd50547ec0e None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:40.881339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea7755-1328-4642-8d53-cfd50547ec0e None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:40.881831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cea7755-1328-4642-8d53-cfd50547ec0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:40.902054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 12/24] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:40.918990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-4cea7755-1328-4642-8d53-cfd50547ec0e None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:40.920606 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4cea7755-1328-4642-8d53-cfd50547ec0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:41.041909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 13/25] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:41.183722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10197b9d-8a79-4363-a581-1bd65c084ede None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:41.183998 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10197b9d-8a79-4363-a581-1bd65c084ede None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:41.184247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10197b9d-8a79-4363-a581-1bd65c084ede None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:41.185661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 13/26] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:41.189557 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b49abe-445d-4d89-b2e8-a4f1a7be86ad None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:41.190090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b49abe-445d-4d89-b2e8-a4f1a7be86ad None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:41.190555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b49abe-445d-4d89-b2e8-a4f1a7be86ad None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:41.191186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fd2b6-1663-4c2c-bd33-1f1c5796f1e7 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:41.191742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fd2b6-1663-4c2c-bd33-1f1c5796f1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:41.192213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fd2b6-1663-4c2c-bd33-1f1c5796f1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:41.192610 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 14/27] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:41.200526 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ccec9-e5db-4b17-b42a-24316db46c00 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:41.201048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ccec9-e5db-4b17-b42a-24316db46c00 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:41.201544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ccec9-e5db-4b17-b42a-24316db46c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:41.223216 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-af2fd2b6-1663-4c2c-bd33-1f1c5796f1e7 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:41.224848 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-af2fd2b6-1663-4c2c-bd33-1f1c5796f1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:41.240513 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-926ccec9-e5db-4b17-b42a-24316db46c00 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:41.244117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-926ccec9-e5db-4b17-b42a-24316db46c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:41.322212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:41.364399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 14/28] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:53:41.366378 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fd2b6-1663-4c2c-bd33-1f1c5796f1e7 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:41.371761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:41.373324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 15/29] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:53:41.478724 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fd2b6-1663-4c2c-bd33-1f1c5796f1e7 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:41.480203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ccec9-e5db-4b17-b42a-24316db46c00 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:41.541005 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57479513-ebf9-4327-a189-33a1e2594283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:41.541662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57479513-ebf9-4327-a189-33a1e2594283 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:41.541936 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57479513-ebf9-4327-a189-33a1e2594283 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:41.542213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57479513-ebf9-4327-a189-33a1e2594283 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:41.568298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57479513-ebf9-4327-a189-33a1e2594283 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-57479513-ebf9-4327-a189-33a1e2594283 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:53:41.573488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:53:41.574311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:53:41.575057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:53:41.575057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:53:41.575057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:53:41.575057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:53:41.575057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:53:41.575057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:53:41.575057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:53:41.575057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:53:41.575057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:53:41.575057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:53:41.575057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:53:41.575057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:53:41.579472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 15/30] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:53:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:53:41.584630 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57479513-ebf9-4327-a189-33a1e2594283 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:41.650232 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ccec9-e5db-4b17-b42a-24316db46c00 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:41.713852 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a64cd-b535-4419-b6f3-1394eea8ecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:41.714499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a64cd-b535-4419-b6f3-1394eea8ecf8 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:41.714776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a64cd-b535-4419-b6f3-1394eea8ecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:41.715047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a64cd-b535-4419-b6f3-1394eea8ecf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:41.735776 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:41.736305 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a64cd-b535-4419-b6f3-1394eea8ecf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:41.737819 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a64cd-b535-4419-b6f3-1394eea8ecf8 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-bf0a64cd-b535-4419-b6f3-1394eea8ecf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:53:41.761345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:53:41.762241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:53:41.763029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:53:41.763029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:53:41.763029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:53:41.763029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 16/31] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:53:41.768454 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a64cd-b535-4419-b6f3-1394eea8ecf8 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:41.796104 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57479513-ebf9-4327-a189-33a1e2594283 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:41.800326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:41.834734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec6f61-6c11-4843-a035-adb3d6150c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:41.835419 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec6f61-6c11-4843-a035-adb3d6150c14 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:41.835732 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec6f61-6c11-4843-a035-adb3d6150c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:41.836007 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec6f61-6c11-4843-a035-adb3d6150c14 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:41.836688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec6f61-6c11-4843-a035-adb3d6150c14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:41.836989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec6f61-6c11-4843-a035-adb3d6150c14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:41.838772 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec6f61-6c11-4843-a035-adb3d6150c14 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:41.847606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 16/32] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:53:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:41.848357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec6f61-6c11-4843-a035-adb3d6150c14 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:41.875387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a64cd-b535-4419-b6f3-1394eea8ecf8 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:41.924142 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd4e82-54c5-47b8-8877-e195802769e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:41.925283 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd4e82-54c5-47b8-8877-e195802769e1 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:41.926466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd4e82-54c5-47b8-8877-e195802769e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:41.926925 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd4e82-54c5-47b8-8877-e195802769e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:41.928024 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd4e82-54c5-47b8-8877-e195802769e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:41.928524 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd4e82-54c5-47b8-8877-e195802769e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:41.931203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd4e82-54c5-47b8-8877-e195802769e1 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:41.946603 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 17/33] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:53:41.947719 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd4e82-54c5-47b8-8877-e195802769e1 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:41.966078 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec6f61-6c11-4843-a035-adb3d6150c14 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:41.995120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:42.005601 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda71e2-e879-4b1c-80ff-6c2ef2a3021a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:42.006354 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda71e2-e879-4b1c-80ff-6c2ef2a3021a None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:42.006728 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda71e2-e879-4b1c-80ff-6c2ef2a3021a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:42.007095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda71e2-e879-4b1c-80ff-6c2ef2a3021a None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:42.008008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda71e2-e879-4b1c-80ff-6c2ef2a3021a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:42.009609 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afda71e2-e879-4b1c-80ff-6c2ef2a3021a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:42.053693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:42.067087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd4e82-54c5-47b8-8877-e195802769e1 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:42.104474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fef221-b7df-44a0-9917-edfcbc3f6455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:42.105166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fef221-b7df-44a0-9917-edfcbc3f6455 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:42.105625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fef221-b7df-44a0-9917-edfcbc3f6455 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:42.105990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fef221-b7df-44a0-9917-edfcbc3f6455 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:42.106765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fef221-b7df-44a0-9917-edfcbc3f6455 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:42.108511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fef221-b7df-44a0-9917-edfcbc3f6455 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:42.156837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 17/34] 10.222.0.27 () {60 vars in 1131 bytes} [Thu Sep 8 15:53:41 2022] POST /identity/v3/roles => generated 219 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:53:42.157892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda71e2-e879-4b1c-80ff-6c2ef2a3021a None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:42.253970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 18/35] 10.222.0.27 () {60 vars in 1131 bytes} [Thu Sep 8 15:53:41 2022] POST /identity/v3/roles => generated 225 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:53:42.260349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fef221-b7df-44a0-9917-edfcbc3f6455 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:42.269532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afda71e2-e879-4b1c-80ff-6c2ef2a3021a None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:42.308873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48109955-0394-4817-8736-4aab9dc8ba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:42.309649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48109955-0394-4817-8736-4aab9dc8ba67 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:42.309935 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48109955-0394-4817-8736-4aab9dc8ba67 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:42.310213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48109955-0394-4817-8736-4aab9dc8ba67 None admin] PATH_INFO: `/v3/users/02fc092547d04cbf984e632c0755920a` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:42.311863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48109955-0394-4817-8736-4aab9dc8ba67 None admin] RBAC: Authorizing `identity:get_user(user_id=02fc092547d04cbf984e632c0755920a)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:42.313730 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48109955-0394-4817-8736-4aab9dc8ba67 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:42.316132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 18/36] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:53:42 2022] GET /identity/v3/users/02fc092547d04cbf984e632c0755920a => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:42.321969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48109955-0394-4817-8736-4aab9dc8ba67 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:42.392778 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:42.442944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fef221-b7df-44a0-9917-edfcbc3f6455 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:42.470801 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48109955-0394-4817-8736-4aab9dc8ba67 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:42.511242 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a04287-3f4c-4c29-986b-6f7fe61f7258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:42.511950 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a04287-3f4c-4c29-986b-6f7fe61f7258 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:42.512234 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a04287-3f4c-4c29-986b-6f7fe61f7258 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:42.512513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a04287-3f4c-4c29-986b-6f7fe61f7258 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:42.513827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a04287-3f4c-4c29-986b-6f7fe61f7258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:42.515590 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a04287-3f4c-4c29-986b-6f7fe61f7258 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:42.517559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 19/37] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:53:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:42.523516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a04287-3f4c-4c29-986b-6f7fe61f7258 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:42.558712 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38376e79-ebf7-4c92-ac3c-9db303914db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:42.559367 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38376e79-ebf7-4c92-ac3c-9db303914db5 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:42.559675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38376e79-ebf7-4c92-ac3c-9db303914db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:42.559944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38376e79-ebf7-4c92-ac3c-9db303914db5 None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:42.560907 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38376e79-ebf7-4c92-ac3c-9db303914db5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:42.563811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38376e79-ebf7-4c92-ac3c-9db303914db5 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:42.612802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:42.623795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 19/38] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Sep 8 15:53:42 2022] POST /identity/v3/projects => generated 301 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:53:42.624935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d469c53-a811-4cc2-8325-c21f85f561ff None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:42.625220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d469c53-a811-4cc2-8325-c21f85f561ff None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:42.625496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d469c53-a811-4cc2-8325-c21f85f561ff None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:42.627275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 20/39] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:42.637777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c78df-f7cf-4283-8732-5fb7ad3f3b0d None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:42.638062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c78df-f7cf-4283-8732-5fb7ad3f3b0d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:42.638340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c78df-f7cf-4283-8732-5fb7ad3f3b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:42.661958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-032c78df-f7cf-4283-8732-5fb7ad3f3b0d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:42.664364 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-032c78df-f7cf-4283-8732-5fb7ad3f3b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:42.685710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a04287-3f4c-4c29-986b-6f7fe61f7258 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:42.719080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:42.741322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fcdd1-1a0f-4c27-b6b9-26f843b2440e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:42.742344 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fcdd1-1a0f-4c27-b6b9-26f843b2440e None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:42.742748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fcdd1-1a0f-4c27-b6b9-26f843b2440e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:42.743188 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fcdd1-1a0f-4c27-b6b9-26f843b2440e None admin] PATH_INFO: `/v3/domains/default/users/02fc092547d04cbf984e632c0755920a/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:42.746033 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3fcdd1-1a0f-4c27-b6b9-26f843b2440e None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=02fc092547d04cbf984e632c0755920a, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:42.748721 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3fcdd1-1a0f-4c27-b6b9-26f843b2440e None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:42.759034 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 21/40] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:42.776352 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 20/41] 10.222.0.27 () {58 vars in 1361 bytes} [Thu Sep 8 15:53:42 2022] PUT /identity/v3/domains/default/users/02fc092547d04cbf984e632c0755920a/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:53:42.966440 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188bf33-5443-4f26-b05a-6853a19102e4 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:42.967377 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188bf33-5443-4f26-b05a-6853a19102e4 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:42.969045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b188bf33-5443-4f26-b05a-6853a19102e4 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:42.970078 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 22/42] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:42.976294 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af3a77-d91d-4c3f-ad94-cea5c8b84b17 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:42.976753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af3a77-d91d-4c3f-ad94-cea5c8b84b17 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:42.976947 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af3a77-d91d-4c3f-ad94-cea5c8b84b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:43.037471 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188bf33-5443-4f26-b05a-6853a19102e4 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:43.106394 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-26af3a77-d91d-4c3f-ad94-cea5c8b84b17 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:43.107934 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-26af3a77-d91d-4c3f-ad94-cea5c8b84b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:43.154160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b188bf33-5443-4f26-b05a-6853a19102e4 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:43.181947 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 21/43] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:43.188150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:43.195159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98054d-7d04-4a9f-9e56-c16a770c2d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:43.195818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98054d-7d04-4a9f-9e56-c16a770c2d85 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:43.196095 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98054d-7d04-4a9f-9e56-c16a770c2d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:43.196364 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b98054d-7d04-4a9f-9e56-c16a770c2d85 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:43.197046 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98054d-7d04-4a9f-9e56-c16a770c2d85 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:43.198559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b98054d-7d04-4a9f-9e56-c16a770c2d85 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:43.214421 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 23/44] 10.222.0.27 () {60 vars in 1131 bytes} [Thu Sep 8 15:53:43 2022] POST /identity/v3/roles => generated 223 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:53:43.448069 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af3a77-d91d-4c3f-ad94-cea5c8b84b17 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:43.551775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af3a77-d91d-4c3f-ad94-cea5c8b84b17 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:43.565649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:43.589578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45522a-3685-490f-8df4-063ed457f51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:43.590222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45522a-3685-490f-8df4-063ed457f51d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:43.590497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45522a-3685-490f-8df4-063ed457f51d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:43.590767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45522a-3685-490f-8df4-063ed457f51d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:43.591921 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45522a-3685-490f-8df4-063ed457f51d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:43.593379 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a45522a-3685-490f-8df4-063ed457f51d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:43.595235 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 22/45] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:53:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:43.600783 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98054d-7d04-4a9f-9e56-c16a770c2d85 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:43.701826 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b98054d-7d04-4a9f-9e56-c16a770c2d85 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:43.738864 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ebae2-fe13-4db5-a34b-ecdd1264bfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:43.739487 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ebae2-fe13-4db5-a34b-ecdd1264bfca None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:43.739790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ebae2-fe13-4db5-a34b-ecdd1264bfca None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:43.740060 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ebae2-fe13-4db5-a34b-ecdd1264bfca None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:43.740724 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ebae2-fe13-4db5-a34b-ecdd1264bfca None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:43.742182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ebae2-fe13-4db5-a34b-ecdd1264bfca None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:43.769093 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 24/46] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Sep 8 15:53:43 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:53:44.997420 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07508a55-078d-46f6-a571-ad8fc49a8e01 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:44.997965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07508a55-078d-46f6-a571-ad8fc49a8e01 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:44.997965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07508a55-078d-46f6-a571-ad8fc49a8e01 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:44.999320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 23/47] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:45.004733 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd636669-905d-4619-90fa-661cdc08cbf8 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:45.005137 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd636669-905d-4619-90fa-661cdc08cbf8 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:45.005298 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd636669-905d-4619-90fa-661cdc08cbf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:45.023716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-cd636669-905d-4619-90fa-661cdc08cbf8 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:45.024912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-cd636669-905d-4619-90fa-661cdc08cbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:45.060366 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:45.094109 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 25/48] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:45.358381 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07508a55-078d-46f6-a571-ad8fc49a8e01 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:45.461554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07508a55-078d-46f6-a571-ad8fc49a8e01 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:45.497098 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba179e2-c27c-4e53-9aa8-2ddc23e89ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:45.497745 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba179e2-c27c-4e53-9aa8-2ddc23e89ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:45.497971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba179e2-c27c-4e53-9aa8-2ddc23e89ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:45.498330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba179e2-c27c-4e53-9aa8-2ddc23e89ae1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:45.508575 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba179e2-c27c-4e53-9aa8-2ddc23e89ae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-bba179e2-c27c-4e53-9aa8-2ddc23e89ae1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:53:45.513461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:53:45.514341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:53:45.515097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:53:45.515097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:53:45.515097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:53:45.515097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:53:45.515097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:53:45.515097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:53:45.515097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:53:45.515097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:53:45.515097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:53:45.515097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:53:45.515097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:53:45.515097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:53:45.515097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 24/49] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:53:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:53:45.521945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd636669-905d-4619-90fa-661cdc08cbf8 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:45.620315 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd636669-905d-4619-90fa-661cdc08cbf8 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:45.654580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c03ea-fd05-4132-b429-e740a794598e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:45.655195 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c03ea-fd05-4132-b429-e740a794598e None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:45.655540 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c03ea-fd05-4132-b429-e740a794598e None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:45.655763 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c03ea-fd05-4132-b429-e740a794598e None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:45.656429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c03ea-fd05-4132-b429-e740a794598e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:45.656730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c03ea-fd05-4132-b429-e740a794598e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:45.658123 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c03ea-fd05-4132-b429-e740a794598e None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:45.670186 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 26/50] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:53:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:45.676695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba179e2-c27c-4e53-9aa8-2ddc23e89ae1 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:45.722537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:45.783603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba179e2-c27c-4e53-9aa8-2ddc23e89ae1 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:45.822196 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f1da7-ab34-461f-ab13-cc1473becd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:45.822845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f1da7-ab34-461f-ab13-cc1473becd69 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:45.823123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f1da7-ab34-461f-ab13-cc1473becd69 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:45.823397 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f1da7-ab34-461f-ab13-cc1473becd69 None admin] PATH_INFO: `/v3/users/02fc092547d04cbf984e632c0755920a` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:45.824608 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f1da7-ab34-461f-ab13-cc1473becd69 None admin] RBAC: Authorizing `identity:get_user(user_id=02fc092547d04cbf984e632c0755920a)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:45.826086 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f1da7-ab34-461f-ab13-cc1473becd69 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:45.827742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 25/51] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:53:45 2022] GET /identity/v3/users/02fc092547d04cbf984e632c0755920a => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:45.833538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c03ea-fd05-4132-b429-e740a794598e None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:45.870956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:45.894827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9ef25-4e31-48d9-aae6-2084b60217e4 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:45.895111 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9ef25-4e31-48d9-aae6-2084b60217e4 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:45.895390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a9ef25-4e31-48d9-aae6-2084b60217e4 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:45.896787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 26/52] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:45.902227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76807476-bfef-4290-a399-e352661d0464 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:45.902479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76807476-bfef-4290-a399-e352661d0464 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:45.902769 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76807476-bfef-4290-a399-e352661d0464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:45.922156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-76807476-bfef-4290-a399-e352661d0464 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:45.923544 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-76807476-bfef-4290-a399-e352661d0464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:45.940489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c03ea-fd05-4132-b429-e740a794598e None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:45.977061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d09c1c-ba7e-419f-9850-850fc2b4edea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:45.977715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d09c1c-ba7e-419f-9850-850fc2b4edea None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:45.977992 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d09c1c-ba7e-419f-9850-850fc2b4edea None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:45.978261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d09c1c-ba7e-419f-9850-850fc2b4edea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:45.979395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d09c1c-ba7e-419f-9850-850fc2b4edea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:45.980893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d09c1c-ba7e-419f-9850-850fc2b4edea None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:45.982472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 27/53] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:53:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:45.988382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d09c1c-ba7e-419f-9850-850fc2b4edea None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:45.999337 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 27/54] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:53:46.198364 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d09c1c-ba7e-419f-9850-850fc2b4edea None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:46.200068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:46.242398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ad453-d87c-4f12-8321-fdcc71ab91da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:46.243945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ad453-d87c-4f12-8321-fdcc71ab91da None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:46.244248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ad453-d87c-4f12-8321-fdcc71ab91da None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:46.244524 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ad453-d87c-4f12-8321-fdcc71ab91da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:46.245265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ad453-d87c-4f12-8321-fdcc71ab91da None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.domain.id=default, user.id=02fc092547d04cbf984e632c0755920a) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:46.245576 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ad453-d87c-4f12-8321-fdcc71ab91da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:46.249617 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ad453-d87c-4f12-8321-fdcc71ab91da None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:46.262116 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 28/55] 10.222.0.27 () {56 vars in 1322 bytes} [Thu Sep 8 15:53:45 2022] GET /identity/v3/role_assignments?user.id=02fc092547d04cbf984e632c0755920a&scope.domain.id=default&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 530 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:53:46.262368 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76807476-bfef-4290-a399-e352661d0464 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:46.307561 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:46.375002 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76807476-bfef-4290-a399-e352661d0464 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:46.412615 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2e15fb-2d4e-4d67-86ff-bd38a0edee60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:46.413272 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e15fb-2d4e-4d67-86ff-bd38a0edee60 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:46.413525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e15fb-2d4e-4d67-86ff-bd38a0edee60 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:46.413824 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2e15fb-2d4e-4d67-86ff-bd38a0edee60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:46.414978 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e15fb-2d4e-4d67-86ff-bd38a0edee60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:46.416497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2e15fb-2d4e-4d67-86ff-bd38a0edee60 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:46.418306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 28/56] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:53:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:53:46.424071 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ad453-d87c-4f12-8321-fdcc71ab91da None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:46.527899 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ad453-d87c-4f12-8321-fdcc71ab91da None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:46.567694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f0b9b-dc60-486f-837a-edf385c3e870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:46.568320 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f0b9b-dc60-486f-837a-edf385c3e870 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:46.568600 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f0b9b-dc60-486f-837a-edf385c3e870 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:46.568871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f0b9b-dc60-486f-837a-edf385c3e870 None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:46.569537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f0b9b-dc60-486f-837a-edf385c3e870 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:46.570991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550f0b9b-dc60-486f-837a-edf385c3e870 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:46.600757 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 29/57] 10.222.0.27 () {60 vars in 1140 bytes} [Thu Sep 8 15:53:46 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:53:48.712357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a07119-720b-40fc-a8ec-37e0d9aa2cd6 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:48.712913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a07119-720b-40fc-a8ec-37e0d9aa2cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:48.712913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a07119-720b-40fc-a8ec-37e0d9aa2cd6 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:48.714605 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 29/58] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:48.723282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2448b-e4c7-4caa-bc09-1f83b55058b8 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:48.723772 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2448b-e4c7-4caa-bc09-1f83b55058b8 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:48.723857 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2448b-e4c7-4caa-bc09-1f83b55058b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:48.744398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-e4f2448b-e4c7-4caa-bc09-1f83b55058b8 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:48.745625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e4f2448b-e4c7-4caa-bc09-1f83b55058b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:48.760637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:48.834602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 30/59] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:49.101041 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a07119-720b-40fc-a8ec-37e0d9aa2cd6 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:49.211884 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a07119-720b-40fc-a8ec-37e0d9aa2cd6 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:49.219455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:49.248784 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e52c0-a648-4423-8e42-00e95f4b064a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:49.249413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e52c0-a648-4423-8e42-00e95f4b064a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:49.249688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e52c0-a648-4423-8e42-00e95f4b064a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:49.249956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e52c0-a648-4423-8e42-00e95f4b064a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:49.251120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e52c0-a648-4423-8e42-00e95f4b064a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:49.252629 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e52c0-a648-4423-8e42-00e95f4b064a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:49.254289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 30/60] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:53:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:49.260106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2448b-e4c7-4caa-bc09-1f83b55058b8 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:49.361793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2448b-e4c7-4caa-bc09-1f83b55058b8 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:49.396551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317bede3-56df-4edc-af84-3722c8c265d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:49.397153 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bede3-56df-4edc-af84-3722c8c265d7 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:49.397448 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bede3-56df-4edc-af84-3722c8c265d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:49.397713 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bede3-56df-4edc-af84-3722c8c265d7 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:49.398364 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317bede3-56df-4edc-af84-3722c8c265d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:49.399830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317bede3-56df-4edc-af84-3722c8c265d7 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:49.428484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 31/61] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:53:49 2022] POST /identity/v3/users => generated 275 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:53:52.821364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93052b-613d-47a6-ba72-f39ea4959dd1 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:52.821885 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93052b-613d-47a6-ba72-f39ea4959dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:52.821962 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93052b-613d-47a6-ba72-f39ea4959dd1 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:52.823282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 31/62] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:52.828759 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757507f-52ac-4350-8998-fd7a3c7a6315 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:52.829153 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757507f-52ac-4350-8998-fd7a3c7a6315 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:52.832035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1757507f-52ac-4350-8998-fd7a3c7a6315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:52.838949 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:52.856465 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-1757507f-52ac-4350-8998-fd7a3c7a6315 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:52.857678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1757507f-52ac-4350-8998-fd7a3c7a6315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:52.996527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bd4a4-957b-423f-b8da-8ae10982dee0 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:52.996807 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bd4a4-957b-423f-b8da-8ae10982dee0 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:52.997323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bd4a4-957b-423f-b8da-8ae10982dee0 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:52.998853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 32/63] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:53.001477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 32/64] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:53.021022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bc61f-7104-4e27-a309-435c0a526892 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:53.021622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bc61f-7104-4e27-a309-435c0a526892 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:53.022138 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bc61f-7104-4e27-a309-435c0a526892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:53.056625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-db4bc61f-7104-4e27-a309-435c0a526892 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:53.057941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-db4bc61f-7104-4e27-a309-435c0a526892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:53.189813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e7e80-a9b0-45e3-acf6-0ec220a097b6 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:53.190369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e7e80-a9b0-45e3-acf6-0ec220a097b6 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:53.190837 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e7e80-a9b0-45e3-acf6-0ec220a097b6 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:53.192544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 33/65] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:53.193965 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d97e3-7fe1-475c-9b28-562c6c02c620 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:53.194437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d97e3-7fe1-475c-9b28-562c6c02c620 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:53.194898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d97e3-7fe1-475c-9b28-562c6c02c620 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:53.196339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 34/66] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:53:53.198481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46318bea-c814-421f-83b8-7d252ee2cff0 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:53.198966 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46318bea-c814-421f-83b8-7d252ee2cff0 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:53.199424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46318bea-c814-421f-83b8-7d252ee2cff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:53.204847 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 33/67] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:53.210516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0e92f-ea8e-4147-be1e-ce309ae07549 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:53.210516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0e92f-ea8e-4147-be1e-ce309ae07549 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:53.210516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0e92f-ea8e-4147-be1e-ce309ae07549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:53.229869 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-bbb0e92f-ea8e-4147-be1e-ce309ae07549 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:53.231057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-bbb0e92f-ea8e-4147-be1e-ce309ae07549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:53.236784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-46318bea-c814-421f-83b8-7d252ee2cff0 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:53.237982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-46318bea-c814-421f-83b8-7d252ee2cff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:53.344247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 35/68] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:53.389187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 34/69] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:53.391516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46318bea-c814-421f-83b8-7d252ee2cff0 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:53.458072 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:53.495987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46318bea-c814-421f-83b8-7d252ee2cff0 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:53.530102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17311793-ebaf-47d0-95a0-90955e1c715c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:53.530755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17311793-ebaf-47d0-95a0-90955e1c715c None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:53.531032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17311793-ebaf-47d0-95a0-90955e1c715c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:53.531438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17311793-ebaf-47d0-95a0-90955e1c715c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:53.533601 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c2286-3e6a-46c2-9fa3-95cd95097fdb None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:53.533892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c2286-3e6a-46c2-9fa3-95cd95097fdb None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:53.534164 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c2286-3e6a-46c2-9fa3-95cd95097fdb None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:53.535687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 35/70] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:53:53.541211 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64775242-7724-4fc7-aea0-30295a7d34a3 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:53.541496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64775242-7724-4fc7-aea0-30295a7d34a3 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:53.541771 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64775242-7724-4fc7-aea0-30295a7d34a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:53.543810 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17311793-ebaf-47d0-95a0-90955e1c715c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-17311793-ebaf-47d0-95a0-90955e1c715c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:53:53.551739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:53:53.552611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:53:53.553404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:53:53.553404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:53:53.553404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:53:53.553404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:53:53.553404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:53:53.553404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:53:53.553404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:53:53.553404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:53:53.553404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:53:53.553404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:53:53.553404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:53:53.553404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:53:53.553404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 36/71] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:53:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:53:53.556216 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17311793-ebaf-47d0-95a0-90955e1c715c None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:53.564582 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-64775242-7724-4fc7-aea0-30295a7d34a3 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:53.565769 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-64775242-7724-4fc7-aea0-30295a7d34a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:53.601287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:53.650785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 36/72] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:53:53.652073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64775242-7724-4fc7-aea0-30295a7d34a3 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:53.689861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17311793-ebaf-47d0-95a0-90955e1c715c None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:53.716194 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:53.727877 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655ecdc-7799-4d14-a8db-32c07aeae4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:53.728528 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655ecdc-7799-4d14-a8db-32c07aeae4de None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:53.728916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655ecdc-7799-4d14-a8db-32c07aeae4de None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:53.729187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655ecdc-7799-4d14-a8db-32c07aeae4de None admin] PATH_INFO: `/v3/roles/member` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:53.741217 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655ecdc-7799-4d14-a8db-32c07aeae4de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-f655ecdc-7799-4d14-a8db-32c07aeae4de None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:53:53.747282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:53:53.748237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:53:53.749001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:53:53.749001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:53:53.749001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:53:53.749001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:53:53.749001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:53:53.749001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:53:53.749001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:53:53.749001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:53:53.749001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:53:53.749001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:53:53.749001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:53:53.749001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:53:53.749001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 37/73] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:53:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:53:53.749850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655ecdc-7799-4d14-a8db-32c07aeae4de None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:53.776819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64775242-7724-4fc7-aea0-30295a7d34a3 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:53.811778 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9663d1c-1712-4255-8cd5-97006823bb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:53.812415 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9663d1c-1712-4255-8cd5-97006823bb02 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:53.812696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9663d1c-1712-4255-8cd5-97006823bb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:53.812971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9663d1c-1712-4255-8cd5-97006823bb02 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:53.813655 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9663d1c-1712-4255-8cd5-97006823bb02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:53.814024 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9663d1c-1712-4255-8cd5-97006823bb02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:53.815546 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9663d1c-1712-4255-8cd5-97006823bb02 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:53.829604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 37/74] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:53:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:53.829604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9663d1c-1712-4255-8cd5-97006823bb02 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:53.856399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:53.859621 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655ecdc-7799-4d14-a8db-32c07aeae4de None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:53.896502 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1c37e-3c96-4ef5-a447-b31538623895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:53.897160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1c37e-3c96-4ef5-a447-b31538623895 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:53.897437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1c37e-3c96-4ef5-a447-b31538623895 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:53.897711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1c37e-3c96-4ef5-a447-b31538623895 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:53.909093 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1c37e-3c96-4ef5-a447-b31538623895 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-39b1c37e-3c96-4ef5-a447-b31538623895 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:53:53.914511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:53:53.915416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:53:53.920256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:53:53.920256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:53:53.920256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:53:53.920256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:53:53.920256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:53:53.920256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:53:53.920256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:53:53.920256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:53:53.920256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:53:53.920256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:53:53.920256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:53:53.920256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:53:53.920256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 38/75] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:53:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:53:53.920256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1c37e-3c96-4ef5-a447-b31538623895 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:53.935517 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9663d1c-1712-4255-8cd5-97006823bb02 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:53.964422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:53.968793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2559b50d-4f9e-4ad5-950e-76f1000de46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:53.969378 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559b50d-4f9e-4ad5-950e-76f1000de46c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:53.969658 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559b50d-4f9e-4ad5-950e-76f1000de46c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:53.969932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2559b50d-4f9e-4ad5-950e-76f1000de46c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:53.980148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2559b50d-4f9e-4ad5-950e-76f1000de46c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-2559b50d-4f9e-4ad5-950e-76f1000de46c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:53:53.985484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:53:53.986383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:53:53.987148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:53:53.987148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:53:53.987148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:53:53.987148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:53:53.987148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:53:53.987148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:53:53.987148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:53:53.987148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:53:53.987148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:53:53.987148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:53:53.987148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:53:53.987148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:53:53.987148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 38/76] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:53:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:53:53.987825 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2559b50d-4f9e-4ad5-950e-76f1000de46c None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:54.026380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1c37e-3c96-4ef5-a447-b31538623895 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:54.059688 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53090954-1a64-45c2-8305-27cc68787618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:54.060310 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53090954-1a64-45c2-8305-27cc68787618 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:54.060594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53090954-1a64-45c2-8305-27cc68787618 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:54.060870 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53090954-1a64-45c2-8305-27cc68787618 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:54.061555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53090954-1a64-45c2-8305-27cc68787618 None admin] RBAC: Adding query filter params (name=member) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:54.061866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53090954-1a64-45c2-8305-27cc68787618 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:54.063307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53090954-1a64-45c2-8305-27cc68787618 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:54.071540 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 39/77] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:53:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:54.072183 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53090954-1a64-45c2-8305-27cc68787618 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:54.074206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:54.091176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2559b50d-4f9e-4ad5-950e-76f1000de46c None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:54.124644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314f947-3fad-4265-985d-c52e410dc7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:54.125287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314f947-3fad-4265-985d-c52e410dc7b3 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:54.125562 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314f947-3fad-4265-985d-c52e410dc7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:54.125840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314f947-3fad-4265-985d-c52e410dc7b3 None admin] PATH_INFO: `/v3/users/02fc092547d04cbf984e632c0755920a` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:54.127039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314f947-3fad-4265-985d-c52e410dc7b3 None admin] RBAC: Authorizing `identity:get_user(user_id=02fc092547d04cbf984e632c0755920a)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:54.128568 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314f947-3fad-4265-985d-c52e410dc7b3 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:54.130168 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 39/78] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:53:53 2022] GET /identity/v3/users/02fc092547d04cbf984e632c0755920a => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:54.130934 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314f947-3fad-4265-985d-c52e410dc7b3 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:54.176544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53090954-1a64-45c2-8305-27cc68787618 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:54.184361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:54.210348 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba835e-90ce-4be7-9bef-a4da311fe38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:54.210985 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba835e-90ce-4be7-9bef-a4da311fe38b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:54.211263 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba835e-90ce-4be7-9bef-a4da311fe38b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:54.211564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba835e-90ce-4be7-9bef-a4da311fe38b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:54.212677 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba835e-90ce-4be7-9bef-a4da311fe38b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:54.214497 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba835e-90ce-4be7-9bef-a4da311fe38b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:54.216128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 40/79] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:53:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:54.217423 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba835e-90ce-4be7-9bef-a4da311fe38b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:54.231526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314f947-3fad-4265-985d-c52e410dc7b3 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:54.264738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea40f21-c71e-4c32-ad41-8510db182a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:54.265355 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea40f21-c71e-4c32-ad41-8510db182a7f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:54.265631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea40f21-c71e-4c32-ad41-8510db182a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:54.265911 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea40f21-c71e-4c32-ad41-8510db182a7f None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:54.266585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea40f21-c71e-4c32-ad41-8510db182a7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:54.266896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea40f21-c71e-4c32-ad41-8510db182a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:54.268309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea40f21-c71e-4c32-ad41-8510db182a7f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:54.276194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 40/80] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:53:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:54.277007 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea40f21-c71e-4c32-ad41-8510db182a7f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:54.294330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:54.316684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba835e-90ce-4be7-9bef-a4da311fe38b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:54.348757 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd80df-f015-4d93-a184-d451ef32378e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:54.349385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd80df-f015-4d93-a184-d451ef32378e None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:54.349642 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd80df-f015-4d93-a184-d451ef32378e None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:54.349919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd80df-f015-4d93-a184-d451ef32378e None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:54.350580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd80df-f015-4d93-a184-d451ef32378e None admin] RBAC: Adding query filter params (name=member) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:54.350875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd80df-f015-4d93-a184-d451ef32378e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:54.352306 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd80df-f015-4d93-a184-d451ef32378e None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:54.360077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 41/81] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:53:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:54.360855 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd80df-f015-4d93-a184-d451ef32378e None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:54.377898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea40f21-c71e-4c32-ad41-8510db182a7f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:54.394367 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:54.413520 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe578d-7de0-4567-98bd-0479b4ba9d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:54.414166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe578d-7de0-4567-98bd-0479b4ba9d7f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:54.414446 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe578d-7de0-4567-98bd-0479b4ba9d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:54.414747 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe578d-7de0-4567-98bd-0479b4ba9d7f None admin] PATH_INFO: `/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:54.428324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe578d-7de0-4567-98bd-0479b4ba9d7f None admin] RBAC: Authorizing `identity:get_user(user_id=8ee93dd82ab0424e9e7c9bdcc0d19a89)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:54.429812 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe578d-7de0-4567-98bd-0479b4ba9d7f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:54.431463 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 41/82] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:53:54 2022] GET /identity/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:54.432225 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe578d-7de0-4567-98bd-0479b4ba9d7f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:54.458742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd80df-f015-4d93-a184-d451ef32378e None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:54.491828 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b4855-a3b5-4f2c-ab2d-ba78426b6fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:54.492364 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b4855-a3b5-4f2c-ab2d-ba78426b6fbe None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:54.492644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b4855-a3b5-4f2c-ab2d-ba78426b6fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:54.492919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b4855-a3b5-4f2c-ab2d-ba78426b6fbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:54.504474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:54.529422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe578d-7de0-4567-98bd-0479b4ba9d7f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:54.561716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b4855-a3b5-4f2c-ab2d-ba78426b6fbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:54.562640 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13bd47-cb72-4465-b394-1aa1878e87ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:54.563193 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497b4855-a3b5-4f2c-ab2d-ba78426b6fbe None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:54.563282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13bd47-cb72-4465-b394-1aa1878e87ac None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:54.563550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13bd47-cb72-4465-b394-1aa1878e87ac None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:54.563825 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13bd47-cb72-4465-b394-1aa1878e87ac None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:54.564479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13bd47-cb72-4465-b394-1aa1878e87ac None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:54.565922 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e13bd47-cb72-4465-b394-1aa1878e87ac None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:54.595388 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 42/83] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:53:54 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:53:54.596301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13bd47-cb72-4465-b394-1aa1878e87ac None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:54.602960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:54.687325 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 42/84] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:53:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:53:54.688559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b4855-a3b5-4f2c-ab2d-ba78426b6fbe None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:54.712910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13bd47-cb72-4465-b394-1aa1878e87ac None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:54.746730 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:54.755272 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f63aca-f915-4ae4-a7d1-aa43a52aa7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:54.755981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f63aca-f915-4ae4-a7d1-aa43a52aa7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:54.756263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f63aca-f915-4ae4-a7d1-aa43a52aa7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:54.756539 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f63aca-f915-4ae4-a7d1-aa43a52aa7d8 None admin] PATH_INFO: `/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:54.757767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f63aca-f915-4ae4-a7d1-aa43a52aa7d8 None admin] RBAC: Authorizing `identity:get_user(user_id=8ee93dd82ab0424e9e7c9bdcc0d19a89)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:54.759268 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f63aca-f915-4ae4-a7d1-aa43a52aa7d8 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:54.760973 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 43/85] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:53:54 2022] GET /identity/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:54.764380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f63aca-f915-4ae4-a7d1-aa43a52aa7d8 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:54.820659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b4855-a3b5-4f2c-ab2d-ba78426b6fbe None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:54.867887 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849f821-fc57-49e1-a720-11d13bde3174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:54.868589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849f821-fc57-49e1-a720-11d13bde3174 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:54.869157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849f821-fc57-49e1-a720-11d13bde3174 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:54.869157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849f821-fc57-49e1-a720-11d13bde3174 None admin] PATH_INFO: `/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:54.870462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849f821-fc57-49e1-a720-11d13bde3174 None admin] RBAC: Authorizing `identity:get_user(user_id=8ee93dd82ab0424e9e7c9bdcc0d19a89)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:54.872057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849f821-fc57-49e1-a720-11d13bde3174 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:54.873675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 43/86] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:53:54 2022] GET /identity/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:54.874438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849f821-fc57-49e1-a720-11d13bde3174 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:54.880800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:54.886389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f63aca-f915-4ae4-a7d1-aa43a52aa7d8 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:54.931090 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea03426-2db8-424f-b0c6-04b5b9ae2b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:54.932095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea03426-2db8-424f-b0c6-04b5b9ae2b57 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:54.932589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea03426-2db8-424f-b0c6-04b5b9ae2b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:54.933029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea03426-2db8-424f-b0c6-04b5b9ae2b57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:54.998999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849f821-fc57-49e1-a720-11d13bde3174 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:55.008303 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea03426-2db8-424f-b0c6-04b5b9ae2b57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:55.009713 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea03426-2db8-424f-b0c6-04b5b9ae2b57 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:55.012025 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:55.046550 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2edbe3-195c-4fea-88eb-b68adfcee073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:55.047384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2edbe3-195c-4fea-88eb-b68adfcee073 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:55.047723 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2edbe3-195c-4fea-88eb-b68adfcee073 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:55.048001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2edbe3-195c-4fea-88eb-b68adfcee073 None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:55.050641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2edbe3-195c-4fea-88eb-b68adfcee073 None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:55.053053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2edbe3-195c-4fea-88eb-b68adfcee073 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:55.055954 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 44/87] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:53:54 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:55.057075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2edbe3-195c-4fea-88eb-b68adfcee073 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:55.118006 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 44/88] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:53:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:53:55.119040 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea03426-2db8-424f-b0c6-04b5b9ae2b57 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:55.154037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:55.195001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2edbe3-195c-4fea-88eb-b68adfcee073 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:55.233006 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:55.237693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353090d-6953-4686-b1d9-ec596659b28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:55.237984 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea03426-2db8-424f-b0c6-04b5b9ae2b57 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:55.239079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353090d-6953-4686-b1d9-ec596659b28e None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:55.239526 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353090d-6953-4686-b1d9-ec596659b28e None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:55.240222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f353090d-6953-4686-b1d9-ec596659b28e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:55.274956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd318f81-5598-4907-b593-0556eb6031f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:55.275797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd318f81-5598-4907-b593-0556eb6031f2 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:55.276219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd318f81-5598-4907-b593-0556eb6031f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:55.276533 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd318f81-5598-4907-b593-0556eb6031f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:55.325666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353090d-6953-4686-b1d9-ec596659b28e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:55.328579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f353090d-6953-4686-b1d9-ec596659b28e None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:55.364664 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd318f81-5598-4907-b593-0556eb6031f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:55.367937 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd318f81-5598-4907-b593-0556eb6031f2 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:55.407673 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:55.444406 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 45/89] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:53:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:53:55.445674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353090d-6953-4686-b1d9-ec596659b28e None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:55.509334 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:55.516236 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 45/90] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:53:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:53:55.517314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd318f81-5598-4907-b593-0556eb6031f2 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:55.565119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f353090d-6953-4686-b1d9-ec596659b28e None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:55.605898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b317b72-3843-452b-a4bd-f1861ff5a60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:55.606786 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b317b72-3843-452b-a4bd-f1861ff5a60f None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:55.607200 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b317b72-3843-452b-a4bd-f1861ff5a60f None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:55.607647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b317b72-3843-452b-a4bd-f1861ff5a60f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:55.609039 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b317b72-3843-452b-a4bd-f1861ff5a60f None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e, user.id=02fc092547d04cbf984e632c0755920a) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:55.609492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b317b72-3843-452b-a4bd-f1861ff5a60f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:55.611637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b317b72-3843-452b-a4bd-f1861ff5a60f None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:55.620628 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 46/91] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:53:55 2022] GET /identity/v3/role_assignments?user.id=02fc092547d04cbf984e632c0755920a&scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:55.621826 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b317b72-3843-452b-a4bd-f1861ff5a60f None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:55.637752 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd318f81-5598-4907-b593-0556eb6031f2 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:55.671831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:55.692874 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913cb31-9bf6-41f4-9cc7-14a8d1846473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:55.694467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913cb31-9bf6-41f4-9cc7-14a8d1846473 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:55.695050 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913cb31-9bf6-41f4-9cc7-14a8d1846473 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:55.697722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913cb31-9bf6-41f4-9cc7-14a8d1846473 None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:55.700596 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913cb31-9bf6-41f4-9cc7-14a8d1846473 None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:55.703464 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913cb31-9bf6-41f4-9cc7-14a8d1846473 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:55.711014 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 46/92] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:53:55 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:55.712428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913cb31-9bf6-41f4-9cc7-14a8d1846473 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:55.797806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:55.830039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913cb31-9bf6-41f4-9cc7-14a8d1846473 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:55.868160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ac342-ed97-4cbf-ab8f-bae72021f558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:55.869166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ac342-ed97-4cbf-ab8f-bae72021f558 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:55.869643 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ac342-ed97-4cbf-ab8f-bae72021f558 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:55.870113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ac342-ed97-4cbf-ab8f-bae72021f558 None admin] PATH_INFO: `/v3/projects/782b3b19a4f54723abdbb3b3cf184ef4` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:55.871525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ac342-ed97-4cbf-ab8f-bae72021f558 None admin] RBAC: Authorizing `identity:get_project(project_id=782b3b19a4f54723abdbb3b3cf184ef4)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:55.873184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ac342-ed97-4cbf-ab8f-bae72021f558 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:55.873598 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b317b72-3843-452b-a4bd-f1861ff5a60f None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:55.875770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 47/93] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:53:55 2022] GET /identity/v3/projects/782b3b19a4f54723abdbb3b3cf184ef4 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:55.876728 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ac342-ed97-4cbf-ab8f-bae72021f558 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:55.985244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ac342-ed97-4cbf-ab8f-bae72021f558 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:55.996696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56fa74-7834-4313-be2e-0433a024473b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:55.997371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56fa74-7834-4313-be2e-0433a024473b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:55.997645 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56fa74-7834-4313-be2e-0433a024473b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:55.997919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb56fa74-7834-4313-be2e-0433a024473b None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:55.999075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56fa74-7834-4313-be2e-0433a024473b None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:56.000588 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb56fa74-7834-4313-be2e-0433a024473b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:56.002758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 47/94] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:53:55 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:56.003058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56fa74-7834-4313-be2e-0433a024473b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:56.040011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0a41e-b56a-4d18-b00e-08e845c9b940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:56.040626 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0a41e-b56a-4d18-b00e-08e845c9b940 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:56.040901 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0a41e-b56a-4d18-b00e-08e845c9b940 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:56.044022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f0a41e-b56a-4d18-b00e-08e845c9b940 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:56.045584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0a41e-b56a-4d18-b00e-08e845c9b940 None admin] RBAC: Adding query filter params (role.id=abbd88ef58a544db8ee4261548778980, scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e, user.id=8ee93dd82ab0424e9e7c9bdcc0d19a89) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:56.045908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0a41e-b56a-4d18-b00e-08e845c9b940 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:56.048075 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f0a41e-b56a-4d18-b00e-08e845c9b940 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:56.054509 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:56.063748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 48/95] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:53:55 2022] GET /identity/v3/role_assignments?user.id=8ee93dd82ab0424e9e7c9bdcc0d19a89&scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e&role.id=abbd88ef58a544db8ee4261548778980 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:56.065579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0a41e-b56a-4d18-b00e-08e845c9b940 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:56.208328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb56fa74-7834-4313-be2e-0433a024473b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:56.211181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:56.246908 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efc36a-8752-40f9-93b8-46cfdad0b4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:56.247646 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efc36a-8752-40f9-93b8-46cfdad0b4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:56.247925 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efc36a-8752-40f9-93b8-46cfdad0b4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:56.248192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efc36a-8752-40f9-93b8-46cfdad0b4a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:56.251472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efc36a-8752-40f9-93b8-46cfdad0b4a5 None admin] RBAC: Adding query filter params (role.id=abbd88ef58a544db8ee4261548778980, scope.project.id=782b3b19a4f54723abdbb3b3cf184ef4, user.id=8ee93dd82ab0424e9e7c9bdcc0d19a89) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:56.251472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efc36a-8752-40f9-93b8-46cfdad0b4a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:56.251472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efc36a-8752-40f9-93b8-46cfdad0b4a5 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:56.257630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 48/96] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:53:56 2022] GET /identity/v3/role_assignments?user.id=8ee93dd82ab0424e9e7c9bdcc0d19a89&scope.project.id=782b3b19a4f54723abdbb3b3cf184ef4&role.id=abbd88ef58a544db8ee4261548778980 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:56.311740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f0a41e-b56a-4d18-b00e-08e845c9b940 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:56.368315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d4c9d-6fdf-4295-ac7e-d7a40639e592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:56.369380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4c9d-6fdf-4295-ac7e-d7a40639e592 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:56.369863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4c9d-6fdf-4295-ac7e-d7a40639e592 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:56.370329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4c9d-6fdf-4295-ac7e-d7a40639e592 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:56.371303 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d4c9d-6fdf-4295-ac7e-d7a40639e592 None admin] RBAC: Adding query filter params (role.id=bf98c97818504bc48e3bd1b2aef9d3c3, scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e, user.id=8ee93dd82ab0424e9e7c9bdcc0d19a89) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:56.371842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d4c9d-6fdf-4295-ac7e-d7a40639e592 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:56.373538 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d4c9d-6fdf-4295-ac7e-d7a40639e592 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:56.380644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 49/97] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:53:56 2022] GET /identity/v3/role_assignments?user.id=8ee93dd82ab0424e9e7c9bdcc0d19a89&scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e&role.id=bf98c97818504bc48e3bd1b2aef9d3c3 => generated 259 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:57.285441 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cef38-3b3d-4016-a5a8-bd85d55f469f None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:57.285996 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cef38-3b3d-4016-a5a8-bd85d55f469f None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:57.285996 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5cef38-3b3d-4016-a5a8-bd85d55f469f None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:57.287331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 49/98] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:57.302042 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a51bfb-0319-464e-8a39-2b552e4c081c None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:57.302924 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a51bfb-0319-464e-8a39-2b552e4c081c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:57.303428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a51bfb-0319-464e-8a39-2b552e4c081c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:57.324553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-64a51bfb-0319-464e-8a39-2b552e4c081c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:57.326136 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-64a51bfb-0319-464e-8a39-2b552e4c081c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:57.419265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:57.431504 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 50/99] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:57.565362 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a599e9-b468-448c-840a-0a51fb031361 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:57.565737 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a599e9-b468-448c-840a-0a51fb031361 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:57.565899 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a599e9-b468-448c-840a-0a51fb031361 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:57.567687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 50/100] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:57.573152 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f07876-a1ee-456b-9e38-1a49c87a8fe4 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:57.573680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f07876-a1ee-456b-9e38-1a49c87a8fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:57.574154 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f07876-a1ee-456b-9e38-1a49c87a8fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:57.604482 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-76f07876-a1ee-456b-9e38-1a49c87a8fe4 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:57.606030 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-76f07876-a1ee-456b-9e38-1a49c87a8fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:57.700839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 51/101] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:57.875332 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a599e9-b468-448c-840a-0a51fb031361 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:57.979127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a599e9-b468-448c-840a-0a51fb031361 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:58.016163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9c468-5e89-43df-9223-f59f9845faec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:58.016819 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9c468-5e89-43df-9223-f59f9845faec None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:58.017100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9c468-5e89-43df-9223-f59f9845faec None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:58.017372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9c468-5e89-43df-9223-f59f9845faec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:58.030929 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9c468-5e89-43df-9223-f59f9845faec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-12b9c468-5e89-43df-9223-f59f9845faec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:53:58.036253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:53:58.037126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:53:58.037916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:53:58.037916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:53:58.037916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:53:58.037916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:53:58.037916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:53:58.037916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:53:58.037916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:53:58.037916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:53:58.037916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:53:58.037916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:53:58.037916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:53:58.037916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:53:58.037916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 51/102] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:53:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:53:58.043617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f07876-a1ee-456b-9e38-1a49c87a8fe4 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:58.144133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f07876-a1ee-456b-9e38-1a49c87a8fe4 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:58.190441 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f7502-6803-4a6a-b659-2b1991da6493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:58.191203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f7502-6803-4a6a-b659-2b1991da6493 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:58.191514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f7502-6803-4a6a-b659-2b1991da6493 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:58.191790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4f7502-6803-4a6a-b659-2b1991da6493 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:58.192474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f7502-6803-4a6a-b659-2b1991da6493 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:58.192776 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f7502-6803-4a6a-b659-2b1991da6493 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:58.194174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4f7502-6803-4a6a-b659-2b1991da6493 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:58.195808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:58.203632 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 52/103] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:53:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:58.210041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9c468-5e89-43df-9223-f59f9845faec None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:58.240968 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f7502-6803-4a6a-b659-2b1991da6493 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:58.241619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:58.413755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9c468-5e89-43df-9223-f59f9845faec None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:58.455568 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4f7502-6803-4a6a-b659-2b1991da6493 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:58.456750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e5b93-6bef-4536-8478-1710b1031b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:58.457838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e5b93-6bef-4536-8478-1710b1031b5b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:58.458392 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e5b93-6bef-4536-8478-1710b1031b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:58.458802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e5b93-6bef-4536-8478-1710b1031b5b None admin] PATH_INFO: `/v3/users/5cd6191586a2482f8a9120be284d5761` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:58.493811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e5b93-6bef-4536-8478-1710b1031b5b None admin] RBAC: Authorizing `identity:get_user(user_id=5cd6191586a2482f8a9120be284d5761)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:58.495301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e5b93-6bef-4536-8478-1710b1031b5b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:58.497051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 52/104] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:53:58 2022] GET /identity/v3/users/5cd6191586a2482f8a9120be284d5761 => generated 289 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:58.498242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8b120-e209-4653-8718-f6cab6c6ab21 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:58.498529 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8b120-e209-4653-8718-f6cab6c6ab21 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:58.498809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab8b120-e209-4653-8718-f6cab6c6ab21 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:58.500046 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 53/105] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:58.508979 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8b120-e209-4653-8718-f6cab6c6ab21 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:58.514728 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52f3abf-b12f-4922-9e4f-93bccd748c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:58.515383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f3abf-b12f-4922-9e4f-93bccd748c88 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:58.515692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f3abf-b12f-4922-9e4f-93bccd748c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:58.515967 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52f3abf-b12f-4922-9e4f-93bccd748c88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:58.517816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f3abf-b12f-4922-9e4f-93bccd748c88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:58.519499 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52f3abf-b12f-4922-9e4f-93bccd748c88 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:58.525495 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 53/106] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:53:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:53:58.526832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a5298-2803-4f34-8e63-c9452e249ab1 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:58.527115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a5298-2803-4f34-8e63-c9452e249ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:58.527387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a5298-2803-4f34-8e63-c9452e249ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:58.563988 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-c06a5298-2803-4f34-8e63-c9452e249ab1 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:58.565189 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c06a5298-2803-4f34-8e63-c9452e249ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:58.659842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:58.669491 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:58.697880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 54/107] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:58.698740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a5298-2803-4f34-8e63-c9452e249ab1 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:58.699817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab8b120-e209-4653-8718-f6cab6c6ab21 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:58.739149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a0ad6-1017-4ef5-a626-2e59890420e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:58.739743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a0ad6-1017-4ef5-a626-2e59890420e6 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:58.740036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a0ad6-1017-4ef5-a626-2e59890420e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:58.740315 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a0ad6-1017-4ef5-a626-2e59890420e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:58.811134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a0ad6-1017-4ef5-a626-2e59890420e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:58.812893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a0ad6-1017-4ef5-a626-2e59890420e6 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:58.890156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a5298-2803-4f34-8e63-c9452e249ab1 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:58.919472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:58.923614 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 54/108] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:53:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:53:58.929228 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a0ad6-1017-4ef5-a626-2e59890420e6 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:58.959649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41cba2-7ce3-49a6-ae6b-7860225e3534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:58.960805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41cba2-7ce3-49a6-ae6b-7860225e3534 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:58.962385 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41cba2-7ce3-49a6-ae6b-7860225e3534 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:58.962840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41cba2-7ce3-49a6-ae6b-7860225e3534 None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:58.964899 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41cba2-7ce3-49a6-ae6b-7860225e3534 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:58.968570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41cba2-7ce3-49a6-ae6b-7860225e3534 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:59.030600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 55/109] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Sep 8 15:53:58 2022] POST /identity/v3/projects => generated 302 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:53:59.083612 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a0ad6-1017-4ef5-a626-2e59890420e6 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:59.123189 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb036c-b54c-4184-9314-0a4da671d8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:59.124229 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb036c-b54c-4184-9314-0a4da671d8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:59.124683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb036c-b54c-4184-9314-0a4da671d8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:59.125136 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb036c-b54c-4184-9314-0a4da671d8b0 None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:59.126559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb036c-b54c-4184-9314-0a4da671d8b0 None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:59.128259 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb036c-b54c-4184-9314-0a4da671d8b0 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:59.130305 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 55/110] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:53:58 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:59.136578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41cba2-7ce3-49a6-ae6b-7860225e3534 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:59.180511 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:59.246041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb036c-b54c-4184-9314-0a4da671d8b0 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:59.266252 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41cba2-7ce3-49a6-ae6b-7860225e3534 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:59.324246 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:59.330644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7384454-f1a4-4ac0-a33e-d267de4afa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:59.331280 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7384454-f1a4-4ac0-a33e-d267de4afa4d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:59.331612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7384454-f1a4-4ac0-a33e-d267de4afa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:59.331914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7384454-f1a4-4ac0-a33e-d267de4afa4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:59.332668 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7384454-f1a4-4ac0-a33e-d267de4afa4d None admin] RBAC: Adding query filter params (role.id=a9c50f8878284257bb064b71a6f7870b, scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e, user.id=5cd6191586a2482f8a9120be284d5761) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:59.332988 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7384454-f1a4-4ac0-a33e-d267de4afa4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:59.334468 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7384454-f1a4-4ac0-a33e-d267de4afa4d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:59.344060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 56/111] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:53:59 2022] GET /identity/v3/role_assignments?user.id=5cd6191586a2482f8a9120be284d5761&scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e&role.id=a9c50f8878284257bb064b71a6f7870b => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:59.389941 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb036c-b54c-4184-9314-0a4da671d8b0 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:59.442215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5828b-70b4-4e79-ac53-5faf9b88929c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:59.442850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5828b-70b4-4e79-ac53-5faf9b88929c None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:59.443091 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5828b-70b4-4e79-ac53-5faf9b88929c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:59.443337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5828b-70b4-4e79-ac53-5faf9b88929c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:59.451313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f599a3c-7a56-4198-b856-36e46dda9b42 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:59.451631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f599a3c-7a56-4198-b856-36e46dda9b42 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:59.451917 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f599a3c-7a56-4198-b856-36e46dda9b42 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:59.453294 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 57/112] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:53:59.459058 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fb15f-5591-42ac-a8b4-870dfdc683c4 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:59.459668 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fb15f-5591-42ac-a8b4-870dfdc683c4 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:59.459668 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fb15f-5591-42ac-a8b4-870dfdc683c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:59.480028 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5828b-70b4-4e79-ac53-5faf9b88929c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-ceb5828b-70b4-4e79-ac53-5faf9b88929c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:53:59.485293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:53:59.486220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:53:59.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:53:59.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:53:59.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:53:59.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:53:59.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:53:59.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:53:59.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:53:59.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:53:59.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:53:59.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:53:59.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:53:59.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:53:59.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 56/113] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:53:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:53:59.493478 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-2c7fb15f-5591-42ac-a8b4-870dfdc683c4 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:59.495115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2c7fb15f-5591-42ac-a8b4-870dfdc683c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:59.497131 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5828b-70b4-4e79-ac53-5faf9b88929c None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:59.614963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5828b-70b4-4e79-ac53-5faf9b88929c None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:59.632628 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:59.656527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 58/114] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:53:59.658646 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34bb04-c37d-4672-8a50-a5d85a989f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:59.659884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34bb04-c37d-4672-8a50-a5d85a989f3a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:59.660551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34bb04-c37d-4672-8a50-a5d85a989f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:59.661022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34bb04-c37d-4672-8a50-a5d85a989f3a None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:59.661910 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34bb04-c37d-4672-8a50-a5d85a989f3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:53:59.662400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34bb04-c37d-4672-8a50-a5d85a989f3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:59.664007 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34bb04-c37d-4672-8a50-a5d85a989f3a None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:59.677794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 57/115] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:53:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:53:59.679601 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07ebc3-290d-4d03-ada1-dba574a040ef None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:59.679898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07ebc3-290d-4d03-ada1-dba574a040ef None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:59.680172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07ebc3-290d-4d03-ada1-dba574a040ef None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:59.680558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34bb04-c37d-4672-8a50-a5d85a989f3a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:59.681835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 59/116] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:53:59.689995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392155c-cd00-4bb3-8135-4315604cc112 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:59.690306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392155c-cd00-4bb3-8135-4315604cc112 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:59.690559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392155c-cd00-4bb3-8135-4315604cc112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:59.710374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-e392155c-cd00-4bb3-8135-4315604cc112 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:59.711569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e392155c-cd00-4bb3-8135-4315604cc112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:59.738029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:53:59.789455 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34bb04-c37d-4672-8a50-a5d85a989f3a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:53:59.824850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9308fe-f52c-4f09-bd6b-597dab23aaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:53:59.825853 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9308fe-f52c-4f09-bd6b-597dab23aaa9 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:59.826516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9308fe-f52c-4f09-bd6b-597dab23aaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:59.827240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9308fe-f52c-4f09-bd6b-597dab23aaa9 None admin] PATH_INFO: `/v3/users/02fc092547d04cbf984e632c0755920a` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:59.828938 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9308fe-f52c-4f09-bd6b-597dab23aaa9 None admin] RBAC: Authorizing `identity:get_user(user_id=02fc092547d04cbf984e632c0755920a)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:53:59.829855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 60/117] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:53:59.831190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9308fe-f52c-4f09-bd6b-597dab23aaa9 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:53:59.833221 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 58/118] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:53:59 2022] GET /identity/v3/users/02fc092547d04cbf984e632c0755920a => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:53:59.839350 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392155c-cd00-4bb3-8135-4315604cc112 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:53:59.893112 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd11c1-b8e4-443f-8694-14827b78a9b8 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:59.893725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd11c1-b8e4-443f-8694-14827b78a9b8 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:59.894232 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efd11c1-b8e4-443f-8694-14827b78a9b8 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:59.896300 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 59/119] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:53:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:53:59.915842 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92786288-19af-4240-b447-643ee0926004 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:53:59.916478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92786288-19af-4240-b447-643ee0926004 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:53:59.916941 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92786288-19af-4240-b447-643ee0926004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:53:59.943312 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-92786288-19af-4240-b447-643ee0926004 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:53:59.946724 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-92786288-19af-4240-b447-643ee0926004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:53:59.994281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392155c-cd00-4bb3-8135-4315604cc112 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:00.033557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bb7a6-1e35-4cea-b5ee-06e32adfee1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:00.034587 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bb7a6-1e35-4cea-b5ee-06e32adfee1d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:00.035175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bb7a6-1e35-4cea-b5ee-06e32adfee1d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:00.035687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bb7a6-1e35-4cea-b5ee-06e32adfee1d None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:00.037034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bb7a6-1e35-4cea-b5ee-06e32adfee1d None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:00.038732 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7bb7a6-1e35-4cea-b5ee-06e32adfee1d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:00.040766 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 61/120] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:53:59 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:00.048215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bb7a6-1e35-4cea-b5ee-06e32adfee1d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:00.055331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:00.103422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:00.110249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 60/121] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:00.195831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bb7a6-1e35-4cea-b5ee-06e32adfee1d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:00.226937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92786288-19af-4240-b447-643ee0926004 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:00.236156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb3f3d-18cc-4d82-a136-7a63c3054b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:00.236849 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb3f3d-18cc-4d82-a136-7a63c3054b82 None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:00.237133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb3f3d-18cc-4d82-a136-7a63c3054b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:00.237411 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb3f3d-18cc-4d82-a136-7a63c3054b82 None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e/users/02fc092547d04cbf984e632c0755920a/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:00.239507 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb3f3d-18cc-4d82-a136-7a63c3054b82 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae75fe116ff145cdb11b0b2127b2ff7e, user_id=02fc092547d04cbf984e632c0755920a, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:00.240977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb3f3d-18cc-4d82-a136-7a63c3054b82 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:00.264225 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 62/122] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:54:00 2022] PUT /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e/users/02fc092547d04cbf984e632c0755920a/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:54:00.265000 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb3f3d-18cc-4d82-a136-7a63c3054b82 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:00.347487 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92786288-19af-4240-b447-643ee0926004 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:00.390777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683df01-532c-44cc-9229-144325408418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:00.391469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683df01-532c-44cc-9229-144325408418 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:00.391760 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683df01-532c-44cc-9229-144325408418 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:00.392040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683df01-532c-44cc-9229-144325408418 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:00.405768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683df01-532c-44cc-9229-144325408418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-1683df01-532c-44cc-9229-144325408418 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:00.411383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:00.412307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:00.413053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:00.413053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:00.413053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:00.413053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:00.413053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:00.413053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:00.413053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:00.413053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:00.413053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:00.413053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:00.413053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:00.413053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:00.413053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 61/123] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:54:00.419889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683df01-532c-44cc-9229-144325408418 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:00.459340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:00.542319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683df01-532c-44cc-9229-144325408418 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:00.544638 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:00.561322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb3f3d-18cc-4d82-a136-7a63c3054b82 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:00.589145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7d950-bb03-40d4-96e6-50832aa9f9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:00.589946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7d950-bb03-40d4-96e6-50832aa9f9db None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:00.590225 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7d950-bb03-40d4-96e6-50832aa9f9db None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:00.590850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7d950-bb03-40d4-96e6-50832aa9f9db None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:00.591824 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7d950-bb03-40d4-96e6-50832aa9f9db None admin] RBAC: Adding query filter params (name=member) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:00.592145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7d950-bb03-40d4-96e6-50832aa9f9db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:00.593579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7d950-bb03-40d4-96e6-50832aa9f9db None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:00.605503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 62/124] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:00.607594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6ea51-4e82-4b41-a717-7613b7158f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:00.607594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6ea51-4e82-4b41-a717-7613b7158f32 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:00.607594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6ea51-4e82-4b41-a717-7613b7158f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:00.607594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6ea51-4e82-4b41-a717-7613b7158f32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:00.612554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7d950-bb03-40d4-96e6-50832aa9f9db None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:00.620938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e6ea51-4e82-4b41-a717-7613b7158f32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-54e6ea51-4e82-4b41-a717-7613b7158f32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:00.626165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:00.627271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:00.628305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:00.628305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:00.628305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:00.628305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:00.628305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:00.628305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:00.628305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:00.628305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:00.628305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:00.628305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:00.628305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:00.628305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:00.630756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 63/125] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:00.636110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6ea51-4e82-4b41-a717-7613b7158f32 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:00.725729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:00.732418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7d950-bb03-40d4-96e6-50832aa9f9db None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:00.772309 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b1465-648d-431c-9ae5-c37a05810622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:00.773357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b1465-648d-431c-9ae5-c37a05810622 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:00.773710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b1465-648d-431c-9ae5-c37a05810622 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:00.774054 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b1465-648d-431c-9ae5-c37a05810622 None admin] PATH_INFO: `/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:00.775466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b1465-648d-431c-9ae5-c37a05810622 None admin] RBAC: Authorizing `identity:get_user(user_id=8ee93dd82ab0424e9e7c9bdcc0d19a89)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:00.777894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b1465-648d-431c-9ae5-c37a05810622 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:00.779625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 63/126] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:00 2022] GET /identity/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:00.786145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b1465-648d-431c-9ae5-c37a05810622 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:00.847725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6ea51-4e82-4b41-a717-7613b7158f32 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:00.903004 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b1465-648d-431c-9ae5-c37a05810622 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:00.936181 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23d269-6b88-408b-9aa3-f1c6ae9b5e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:00.937111 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23d269-6b88-408b-9aa3-f1c6ae9b5e99 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:00.937571 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23d269-6b88-408b-9aa3-f1c6ae9b5e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:00.938022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23d269-6b88-408b-9aa3-f1c6ae9b5e99 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:00.938881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23d269-6b88-408b-9aa3-f1c6ae9b5e99 None admin] RBAC: Adding query filter params (name=member) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:00.939389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23d269-6b88-408b-9aa3-f1c6ae9b5e99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:00.940976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0c92f-b2ec-469e-9149-d5b2e675f643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:00.941237 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23d269-6b88-408b-9aa3-f1c6ae9b5e99 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:00.941751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0c92f-b2ec-469e-9149-d5b2e675f643 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:00.942098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0c92f-b2ec-469e-9149-d5b2e675f643 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:00.942440 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0c92f-b2ec-469e-9149-d5b2e675f643 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:00.957961 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec0c92f-b2ec-469e-9149-d5b2e675f643 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:00.959886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 64/127] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:00.960916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23d269-6b88-408b-9aa3-f1c6ae9b5e99 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-5ec0c92f-b2ec-469e-9149-d5b2e675f643 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:00.964076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:00.964967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:00.965743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:00.965743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:00.965743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:00.965743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:00.965743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:00.965743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:00.965743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:00.965743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:00.965743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:00.965743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:00.965743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:00.965743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:00.965743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 64/128] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:54:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:00.967999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0c92f-b2ec-469e-9149-d5b2e675f643 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:00.986084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:00.997452 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:01.076664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec0c92f-b2ec-469e-9149-d5b2e675f643 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:01.112919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0976e6a-efc6-4e90-b106-cfbdd52ca800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:01.113678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0976e6a-efc6-4e90-b106-cfbdd52ca800 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:01.114076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0976e6a-efc6-4e90-b106-cfbdd52ca800 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:01.117833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0976e6a-efc6-4e90-b106-cfbdd52ca800 None admin] PATH_INFO: `/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:01.117833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0976e6a-efc6-4e90-b106-cfbdd52ca800 None admin] RBAC: Authorizing `identity:get_user(user_id=8ee93dd82ab0424e9e7c9bdcc0d19a89)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:01.120298 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0976e6a-efc6-4e90-b106-cfbdd52ca800 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:01.121917 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 65/129] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:00 2022] GET /identity/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:01.123699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0976e6a-efc6-4e90-b106-cfbdd52ca800 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:01.177775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23d269-6b88-408b-9aa3-f1c6ae9b5e99 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:01.234269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:01.242168 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f12c23-a623-469e-8662-afd5bf044a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:01.242824 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f12c23-a623-469e-8662-afd5bf044a98 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:01.243102 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f12c23-a623-469e-8662-afd5bf044a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:01.243375 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f12c23-a623-469e-8662-afd5bf044a98 None admin] PATH_INFO: `/v3/projects/782b3b19a4f54723abdbb3b3cf184ef4` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:01.248166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0976e6a-efc6-4e90-b106-cfbdd52ca800 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:01.250135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f12c23-a623-469e-8662-afd5bf044a98 None admin] RBAC: Authorizing `identity:get_project(project_id=782b3b19a4f54723abdbb3b3cf184ef4)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:01.251652 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f12c23-a623-469e-8662-afd5bf044a98 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:01.254404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 65/130] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:00 2022] GET /identity/v3/projects/782b3b19a4f54723abdbb3b3cf184ef4 => generated 312 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:01.255171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f12c23-a623-469e-8662-afd5bf044a98 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:01.314012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7674428-5c39-4f78-8101-939cd58381b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:01.315426 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7674428-5c39-4f78-8101-939cd58381b7 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:01.315900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7674428-5c39-4f78-8101-939cd58381b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:01.316370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7674428-5c39-4f78-8101-939cd58381b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:01.317198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7674428-5c39-4f78-8101-939cd58381b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:01.317789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7674428-5c39-4f78-8101-939cd58381b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:01.320262 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7674428-5c39-4f78-8101-939cd58381b7 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:01.345008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 66/131] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:01.346054 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7674428-5c39-4f78-8101-939cd58381b7 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:01.429682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:01.459813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7674428-5c39-4f78-8101-939cd58381b7 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:01.497503 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f12c23-a623-469e-8662-afd5bf044a98 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:01.500050 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95abd96-f5b7-40e4-bdd5-df8174d9820c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:01.500730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95abd96-f5b7-40e4-bdd5-df8174d9820c None admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:01.501060 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95abd96-f5b7-40e4-bdd5-df8174d9820c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:01.501332 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95abd96-f5b7-40e4-bdd5-df8174d9820c None admin] PATH_INFO: `/v3/projects/782b3b19a4f54723abdbb3b3cf184ef4/users/8ee93dd82ab0424e9e7c9bdcc0d19a89/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:01.504252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95abd96-f5b7-40e4-bdd5-df8174d9820c None admin] RBAC: Authorizing `identity:create_grant(project_id=782b3b19a4f54723abdbb3b3cf184ef4, user_id=8ee93dd82ab0424e9e7c9bdcc0d19a89, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:01.505714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95abd96-f5b7-40e4-bdd5-df8174d9820c None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:01.535854 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 67/132] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:54:01 2022] PUT /identity/v3/projects/782b3b19a4f54723abdbb3b3cf184ef4/users/8ee93dd82ab0424e9e7c9bdcc0d19a89/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:54:01.536861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95abd96-f5b7-40e4-bdd5-df8174d9820c None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:01.567883 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e0975-da58-415f-9efb-9dd90c862bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:01.568545 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e0975-da58-415f-9efb-9dd90c862bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:01.568836 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e0975-da58-415f-9efb-9dd90c862bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:01.569218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e0975-da58-415f-9efb-9dd90c862bf6 None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:01.570721 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e0975-da58-415f-9efb-9dd90c862bf6 None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:01.572377 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e0975-da58-415f-9efb-9dd90c862bf6 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:01.573906 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:01.574677 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 66/133] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:01 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:01.580976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e0975-da58-415f-9efb-9dd90c862bf6 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:01.777810 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95abd96-f5b7-40e4-bdd5-df8174d9820c None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:01.803550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e0975-da58-415f-9efb-9dd90c862bf6 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:01.851280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81507-ecd6-478b-bee9-821e26cc763a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:01.851982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81507-ecd6-478b-bee9-821e26cc763a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:01.852265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81507-ecd6-478b-bee9-821e26cc763a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:01.852537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe81507-ecd6-478b-bee9-821e26cc763a None admin] PATH_INFO: `/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:01.853728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe81507-ecd6-478b-bee9-821e26cc763a None admin] RBAC: Authorizing `identity:get_user(user_id=8ee93dd82ab0424e9e7c9bdcc0d19a89)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:01.855212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe81507-ecd6-478b-bee9-821e26cc763a None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:01.857145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 68/134] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:01 2022] GET /identity/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89 => generated 275 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:01.864465 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81507-ecd6-478b-bee9-821e26cc763a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:01.896555 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a2a37-7721-476b-8ef2-d91ea9535055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:01.897283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a2a37-7721-476b-8ef2-d91ea9535055 None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:01.897559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a2a37-7721-476b-8ef2-d91ea9535055 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:01.897834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a2a37-7721-476b-8ef2-d91ea9535055 None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e/users/8ee93dd82ab0424e9e7c9bdcc0d19a89/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:01.900532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a2a37-7721-476b-8ef2-d91ea9535055 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae75fe116ff145cdb11b0b2127b2ff7e, user_id=8ee93dd82ab0424e9e7c9bdcc0d19a89, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:01.902003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7a2a37-7721-476b-8ef2-d91ea9535055 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:01.916910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 67/135] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:54:01 2022] PUT /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e/users/8ee93dd82ab0424e9e7c9bdcc0d19a89/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 15:54:01.918384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386176b0-c9a8-46d7-93bb-4294887d1c3f None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:01.918802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386176b0-c9a8-46d7-93bb-4294887d1c3f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:01.919093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386176b0-c9a8-46d7-93bb-4294887d1c3f None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:01.920528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 68/136] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:01.933179 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358d073-0171-4af2-a10f-79b5b5fcbf9d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:01.933560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358d073-0171-4af2-a10f-79b5b5fcbf9d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:01.933840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358d073-0171-4af2-a10f-79b5b5fcbf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:01.943629 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:01.965146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:01.971017 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-b358d073-0171-4af2-a10f-79b5b5fcbf9d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:01.973346 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b358d073-0171-4af2-a10f-79b5b5fcbf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:01.988275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe81507-ecd6-478b-bee9-821e26cc763a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:02.036277 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98217a-53a9-4570-8ea9-b4e5e78fa0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:02.036865 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98217a-53a9-4570-8ea9-b4e5e78fa0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:02.037144 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98217a-53a9-4570-8ea9-b4e5e78fa0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:02.037421 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b98217a-53a9-4570-8ea9-b4e5e78fa0b5 None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:02.044983 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98217a-53a9-4570-8ea9-b4e5e78fa0b5 None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:02.047384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b98217a-53a9-4570-8ea9-b4e5e78fa0b5 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:02.050080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 69/137] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:01 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:02.057370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98217a-53a9-4570-8ea9-b4e5e78fa0b5 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:02.097617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 69/138] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:02.200113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b98217a-53a9-4570-8ea9-b4e5e78fa0b5 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:02.235316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696db2d1-3fe8-43e7-a222-f24181072bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:02.236033 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696db2d1-3fe8-43e7-a222-f24181072bbf None admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:02.236313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696db2d1-3fe8-43e7-a222-f24181072bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:02.236592 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696db2d1-3fe8-43e7-a222-f24181072bbf None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e/users/8ee93dd82ab0424e9e7c9bdcc0d19a89/roles/bf98c97818504bc48e3bd1b2aef9d3c3` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:02.239023 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696db2d1-3fe8-43e7-a222-f24181072bbf None admin] RBAC: Authorizing `identity:create_grant(project_id=ae75fe116ff145cdb11b0b2127b2ff7e, user_id=8ee93dd82ab0424e9e7c9bdcc0d19a89, role_id=bf98c97818504bc48e3bd1b2aef9d3c3)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:02.240605 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696db2d1-3fe8-43e7-a222-f24181072bbf None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:02.260329 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 70/139] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:54:02 2022] PUT /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e/users/8ee93dd82ab0424e9e7c9bdcc0d19a89/roles/bf98c97818504bc48e3bd1b2aef9d3c3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:54:02.505132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358d073-0171-4af2-a10f-79b5b5fcbf9d None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:02.515704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090e18c-5e6e-4483-bef4-4d5cdf6addf4 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:02.516332 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090e18c-5e6e-4483-bef4-4d5cdf6addf4 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:02.516332 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090e18c-5e6e-4483-bef4-4d5cdf6addf4 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:02.521064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 71/140] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:02.525053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb40171-08cb-41ae-ba7f-2eef1238d701 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:02.525553 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb40171-08cb-41ae-ba7f-2eef1238d701 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:02.526918 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb40171-08cb-41ae-ba7f-2eef1238d701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:02.562312 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-eeb40171-08cb-41ae-ba7f-2eef1238d701 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:02.565048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-eeb40171-08cb-41ae-ba7f-2eef1238d701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:02.576299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:02.658902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358d073-0171-4af2-a10f-79b5b5fcbf9d None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:02.698479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128b894-1248-485e-ae4b-3e5e39bab69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:02.699193 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128b894-1248-485e-ae4b-3e5e39bab69e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:02.699518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128b894-1248-485e-ae4b-3e5e39bab69e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:02.699796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128b894-1248-485e-ae4b-3e5e39bab69e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:02.703094 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:02.713747 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128b894-1248-485e-ae4b-3e5e39bab69e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-2128b894-1248-485e-ae4b-3e5e39bab69e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:02.719775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:02.720617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:02.721384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:02.721384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:02.721384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:02.721384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:02.721384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:02.721384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:02.721384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:02.721384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:02.721384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:02.721384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:02.721384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:02.721384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:02.724169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 70/141] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:02.730585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128b894-1248-485e-ae4b-3e5e39bab69e None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:02.737308 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 72/142] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:02.836472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128b894-1248-485e-ae4b-3e5e39bab69e None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:02.881164 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e945a8d-222a-4c90-9fe0-fbdbc5558a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:02.882161 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e945a8d-222a-4c90-9fe0-fbdbc5558a6f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:02.882802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e945a8d-222a-4c90-9fe0-fbdbc5558a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:02.883266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e945a8d-222a-4c90-9fe0-fbdbc5558a6f None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:02.884255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e945a8d-222a-4c90-9fe0-fbdbc5558a6f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:02.884813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e945a8d-222a-4c90-9fe0-fbdbc5558a6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:02.886462 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e945a8d-222a-4c90-9fe0-fbdbc5558a6f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:02.899598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 71/143] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:02.905837 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb40171-08cb-41ae-ba7f-2eef1238d701 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:03.060241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb40171-08cb-41ae-ba7f-2eef1238d701 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:03.110099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f509d-a10c-4edc-b3f6-6ecc7f059624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:03.110669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f509d-a10c-4edc-b3f6-6ecc7f059624 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:03.110948 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f509d-a10c-4edc-b3f6-6ecc7f059624 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:03.111220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1f509d-a10c-4edc-b3f6-6ecc7f059624 None admin] PATH_INFO: `/v3/users/5cd6191586a2482f8a9120be284d5761` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:03.112409 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f509d-a10c-4edc-b3f6-6ecc7f059624 None admin] RBAC: Authorizing `identity:get_user(user_id=5cd6191586a2482f8a9120be284d5761)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:03.113975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1f509d-a10c-4edc-b3f6-6ecc7f059624 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:03.115639 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 73/144] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:02 2022] GET /identity/v3/users/5cd6191586a2482f8a9120be284d5761 => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:03.121818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e945a8d-222a-4c90-9fe0-fbdbc5558a6f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:03.225875 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:03.233562 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f509d-a10c-4edc-b3f6-6ecc7f059624 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:03.256159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:03.284392 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e945a8d-222a-4c90-9fe0-fbdbc5558a6f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:03.349410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e2fee-745a-4381-8163-e33b0c493dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:03.350068 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e2fee-745a-4381-8163-e33b0c493dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:03.350339 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e2fee-745a-4381-8163-e33b0c493dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:03.350667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e2fee-745a-4381-8163-e33b0c493dc8 None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:03.352292 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e2fee-745a-4381-8163-e33b0c493dc8 None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:03.353744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e2fee-745a-4381-8163-e33b0c493dc8 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:03.356553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 72/145] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:03 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:03.363650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e2fee-745a-4381-8163-e33b0c493dc8 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:03.421054 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1f509d-a10c-4edc-b3f6-6ecc7f059624 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:03.487940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02281858-53ab-497e-b10f-bb146bb8f4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:03.488978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02281858-53ab-497e-b10f-bb146bb8f4ba None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:03.489441 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02281858-53ab-497e-b10f-bb146bb8f4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:03.489849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02281858-53ab-497e-b10f-bb146bb8f4ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:03.491859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02281858-53ab-497e-b10f-bb146bb8f4ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:03.493509 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02281858-53ab-497e-b10f-bb146bb8f4ba None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:03.499233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 74/146] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:03.506559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02281858-53ab-497e-b10f-bb146bb8f4ba None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:03.550687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e2fee-745a-4381-8163-e33b0c493dc8 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:03.594935 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:03.616450 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:03.634238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9414c-1607-4212-a4dc-e1349b2e8977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:03.634972 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9414c-1607-4212-a4dc-e1349b2e8977 None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:03.635273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9414c-1607-4212-a4dc-e1349b2e8977 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:03.635567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9414c-1607-4212-a4dc-e1349b2e8977 None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e/users/5cd6191586a2482f8a9120be284d5761/roles/a9c50f8878284257bb064b71a6f7870b` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:03.638711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02281858-53ab-497e-b10f-bb146bb8f4ba None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:03.640719 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9414c-1607-4212-a4dc-e1349b2e8977 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae75fe116ff145cdb11b0b2127b2ff7e, user_id=5cd6191586a2482f8a9120be284d5761, role_id=a9c50f8878284257bb064b71a6f7870b)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:03.642189 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9414c-1607-4212-a4dc-e1349b2e8977 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:03.680997 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 73/147] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:54:03 2022] PUT /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e/users/5cd6191586a2482f8a9120be284d5761/roles/a9c50f8878284257bb064b71a6f7870b => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:54:03.682267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6099402-50e1-44ad-909b-e19199bea091 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:03.683969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6099402-50e1-44ad-909b-e19199bea091 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:03.684244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6099402-50e1-44ad-909b-e19199bea091 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:03.685362 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de316c78-e63f-4cd3-9e63-9f8dde2d362d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:03.685987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de316c78-e63f-4cd3-9e63-9f8dde2d362d None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:03.686254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de316c78-e63f-4cd3-9e63-9f8dde2d362d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:03.687719 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 74/148] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:03.687833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de316c78-e63f-4cd3-9e63-9f8dde2d362d None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:03.688457 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de316c78-e63f-4cd3-9e63-9f8dde2d362d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:03.690163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de316c78-e63f-4cd3-9e63-9f8dde2d362d None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:03.693780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3019202-5891-4da1-b0e7-e41f84e16423 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:03.694248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3019202-5891-4da1-b0e7-e41f84e16423 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:03.694663 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3019202-5891-4da1-b0e7-e41f84e16423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:03.730078 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f3019202-5891-4da1-b0e7-e41f84e16423 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:03.735836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 75/149] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:54:03 2022] POST /identity/v3/users => generated 283 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:54:03.736389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f3019202-5891-4da1-b0e7-e41f84e16423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:03.842258 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 75/150] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:04.377596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de316c78-e63f-4cd3-9e63-9f8dde2d362d None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:04.501281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de316c78-e63f-4cd3-9e63-9f8dde2d362d None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:04.538704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:04.545157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d71aa-eb4d-48e0-bf5d-f3008e0db018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:04.546161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d71aa-eb4d-48e0-bf5d-f3008e0db018 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:04.546638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d71aa-eb4d-48e0-bf5d-f3008e0db018 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:04.547154 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d71aa-eb4d-48e0-bf5d-f3008e0db018 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:04.562140 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d71aa-eb4d-48e0-bf5d-f3008e0db018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-107d71aa-eb4d-48e0-bf5d-f3008e0db018 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:04.568240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:04.569205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:04.570110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:04.570110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:04.570110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:04.570110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:04.570110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:04.570110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:04.570110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:04.570110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:04.570110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:04.570110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:04.570110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:54:04.570110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:04.573023 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 76/151] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:54:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:04.578761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3019202-5891-4da1-b0e7-e41f84e16423 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:04.708649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3019202-5891-4da1-b0e7-e41f84e16423 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:04.728919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:04.759015 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff603a-0d35-4774-be9d-89414d7d22af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:04.760074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff603a-0d35-4774-be9d-89414d7d22af None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:04.760539 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff603a-0d35-4774-be9d-89414d7d22af None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:04.761000 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff603a-0d35-4774-be9d-89414d7d22af None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:04.761889 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff603a-0d35-4774-be9d-89414d7d22af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:04.762380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff603a-0d35-4774-be9d-89414d7d22af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:04.764006 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff603a-0d35-4774-be9d-89414d7d22af None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:04.779467 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4a6e2-6838-434c-9c8d-c7a0226e80af None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:04.780404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4a6e2-6838-434c-9c8d-c7a0226e80af None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:04.780877 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4a6e2-6838-434c-9c8d-c7a0226e80af None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:04.782557 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 77/152] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:04.784168 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 76/153] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:04.790330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0d277-feb1-4726-a8c4-e1128510258b None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:04.790869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0d277-feb1-4726-a8c4-e1128510258b None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:04.791342 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0d277-feb1-4726-a8c4-e1128510258b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:04.795241 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff603a-0d35-4774-be9d-89414d7d22af None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:04.825811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-3de0d277-feb1-4726-a8c4-e1128510258b None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:04.827384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3de0d277-feb1-4726-a8c4-e1128510258b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:04.947091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff603a-0d35-4774-be9d-89414d7d22af None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:04.977150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 78/154] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:05.000251 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5a243-5694-4654-81d9-b106639f4466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:05.001321 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5a243-5694-4654-81d9-b106639f4466 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:05.001804 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5a243-5694-4654-81d9-b106639f4466 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:05.002269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5a243-5694-4654-81d9-b106639f4466 None admin] PATH_INFO: `/v3/users/02fc092547d04cbf984e632c0755920a` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:05.003760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be5a243-5694-4654-81d9-b106639f4466 None admin] RBAC: Authorizing `identity:get_user(user_id=02fc092547d04cbf984e632c0755920a)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:05.005519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be5a243-5694-4654-81d9-b106639f4466 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:05.016082 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 77/155] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:04 2022] GET /identity/v3/users/02fc092547d04cbf984e632c0755920a => generated 247 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:05.017627 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0d277-feb1-4726-a8c4-e1128510258b None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:05.178724 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0d277-feb1-4726-a8c4-e1128510258b None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:05.189292 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3ec84-9f3f-43f9-a12e-36f0c21af430 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:05.189907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3ec84-9f3f-43f9-a12e-36f0c21af430 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:05.190379 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3ec84-9f3f-43f9-a12e-36f0c21af430 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:05.192092 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 78/156] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:05.198101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7dec6-0e0a-4244-b051-bd13af6ecc14 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:05.199222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7dec6-0e0a-4244-b051-bd13af6ecc14 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:05.199744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7dec6-0e0a-4244-b051-bd13af6ecc14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:05.225176 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:05.244559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-d0b7dec6-0e0a-4244-b051-bd13af6ecc14 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:05.246108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d0b7dec6-0e0a-4244-b051-bd13af6ecc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:05.255469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003d2e4-a2ba-4cf0-9952-c3a2edb05959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:05.255469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003d2e4-a2ba-4cf0-9952-c3a2edb05959 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:05.255469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003d2e4-a2ba-4cf0-9952-c3a2edb05959 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:05.255469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d003d2e4-a2ba-4cf0-9952-c3a2edb05959 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:05.309412 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:05.354555 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 79/157] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:05.399175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003d2e4-a2ba-4cf0-9952-c3a2edb05959 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:05.400802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d003d2e4-a2ba-4cf0-9952-c3a2edb05959 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:05.455962 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7dec6-0e0a-4244-b051-bd13af6ecc14 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:05.622378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 79/158] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:05.631986 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7dec6-0e0a-4244-b051-bd13af6ecc14 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:05.641428 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003d2e4-a2ba-4cf0-9952-c3a2edb05959 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:05.680096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f6b81-43d3-49f2-a746-1bd16c6c2ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:05.681158 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f6b81-43d3-49f2-a746-1bd16c6c2ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:05.681452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f6b81-43d3-49f2-a746-1bd16c6c2ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:05.681733 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f6b81-43d3-49f2-a746-1bd16c6c2ca4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:05.699256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f6b81-43d3-49f2-a746-1bd16c6c2ca4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-c47f6b81-43d3-49f2-a746-1bd16c6c2ca4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:05.704739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:05.705908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:05.706993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:05.706993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:05.706993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:05.706993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:05.706993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:05.706993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:05.706993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:05.706993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:05.706993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:05.706993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:05.706993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:05.706993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:05.715129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:05.715215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 80/159] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:05.718989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f6b81-43d3-49f2-a746-1bd16c6c2ca4 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:05.776233 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:05.891924 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f6b81-43d3-49f2-a746-1bd16c6c2ca4 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:05.909121 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d003d2e4-a2ba-4cf0-9952-c3a2edb05959 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:05.961654 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87295252-4458-4a04-896f-07a7f611d5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:05.962273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87295252-4458-4a04-896f-07a7f611d5de None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:05.962550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87295252-4458-4a04-896f-07a7f611d5de None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:05.962819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87295252-4458-4a04-896f-07a7f611d5de None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:05.964022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87295252-4458-4a04-896f-07a7f611d5de None admin] RBAC: Adding query filter params (name=member) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:05.964629 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87295252-4458-4a04-896f-07a7f611d5de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:05.966588 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87295252-4458-4a04-896f-07a7f611d5de None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:05.982455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 81/160] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:05.983299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87295252-4458-4a04-896f-07a7f611d5de None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:05.983527 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719eb0e-67fd-40a3-baf2-097b96dc9bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:05.984005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719eb0e-67fd-40a3-baf2-097b96dc9bdd None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:05.984279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719eb0e-67fd-40a3-baf2-097b96dc9bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:05.984551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719eb0e-67fd-40a3-baf2-097b96dc9bdd None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:05.985725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719eb0e-67fd-40a3-baf2-097b96dc9bdd None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:05.987271 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719eb0e-67fd-40a3-baf2-097b96dc9bdd None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:05.989149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 80/161] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:05 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:05.990281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f78fbf-b77f-4456-a8d2-04ce07edc1fd None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:05.990551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f78fbf-b77f-4456-a8d2-04ce07edc1fd None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:05.990822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f78fbf-b77f-4456-a8d2-04ce07edc1fd None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:05.992030 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 81/162] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:05.992718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f78fbf-b77f-4456-a8d2-04ce07edc1fd None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:06.151897 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:06.158697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87295252-4458-4a04-896f-07a7f611d5de None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:06.162551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:06.189576 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f78fbf-b77f-4456-a8d2-04ce07edc1fd None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:06.246212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef64552-b428-4d03-816a-f9cda7d2c333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:06.246862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef64552-b428-4d03-816a-f9cda7d2c333 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:06.247136 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef64552-b428-4d03-816a-f9cda7d2c333 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:06.247405 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef64552-b428-4d03-816a-f9cda7d2c333 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:06.274626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfc428d-e902-4379-845e-c80500cfdc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:06.275327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc428d-e902-4379-845e-c80500cfdc77 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:06.275629 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc428d-e902-4379-845e-c80500cfdc77 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:06.275902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfc428d-e902-4379-845e-c80500cfdc77 None admin] PATH_INFO: `/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:06.276897 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef64552-b428-4d03-816a-f9cda7d2c333 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:06.281713 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc428d-e902-4379-845e-c80500cfdc77 None admin] RBAC: Authorizing `identity:get_user(user_id=8ee93dd82ab0424e9e7c9bdcc0d19a89)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:06.283273 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfc428d-e902-4379-845e-c80500cfdc77 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:06.286228 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 82/163] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:05 2022] GET /identity/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89 => generated 275 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:06.287389 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e64845-2749-4bee-9647-8f306f5f5ce7 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-4ef64552-b428-4d03-816a-f9cda7d2c333 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:06.288580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:06.289424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:06.290186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:06.290186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:06.290186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:06.290186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:06.290186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:06.290186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:06.290186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:06.290186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:06.290186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:06.290186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:06.290186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:06.290186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:06.290769 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e64845-2749-4bee-9647-8f306f5f5ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:06.290769 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e64845-2749-4bee-9647-8f306f5f5ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:06.292261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 82/164] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:06.297689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef64552-b428-4d03-816a-f9cda7d2c333 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:06.329741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-57e64845-2749-4bee-9647-8f306f5f5ce7 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:06.330957 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-57e64845-2749-4bee-9647-8f306f5f5ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:06.443278 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 83/165] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:06.444275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e64845-2749-4bee-9647-8f306f5f5ce7 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:06.517360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef64552-b428-4d03-816a-f9cda7d2c333 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:06.576215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:06.598888 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a89f61-8e55-4a30-b32d-b7f2a198d2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:06.599619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e64845-2749-4bee-9647-8f306f5f5ce7 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:06.600027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a89f61-8e55-4a30-b32d-b7f2a198d2bd None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:06.600469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a89f61-8e55-4a30-b32d-b7f2a198d2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:06.600971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a89f61-8e55-4a30-b32d-b7f2a198d2bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:06.602000 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a89f61-8e55-4a30-b32d-b7f2a198d2bd None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e, user.id=02fc092547d04cbf984e632c0755920a) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:06.602531 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a89f61-8e55-4a30-b32d-b7f2a198d2bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:06.604294 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a89f61-8e55-4a30-b32d-b7f2a198d2bd None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:06.616205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 83/166] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:06 2022] GET /identity/v3/role_assignments?user.id=02fc092547d04cbf984e632c0755920a&scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 608 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:06.617514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a89f61-8e55-4a30-b32d-b7f2a198d2bd None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:06.667041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb735578-3bb5-4a61-a114-a58bd32e1315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:06.667698 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb735578-3bb5-4a61-a114-a58bd32e1315 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:06.668032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb735578-3bb5-4a61-a114-a58bd32e1315 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:06.668306 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb735578-3bb5-4a61-a114-a58bd32e1315 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:06.671819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:06.812097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb735578-3bb5-4a61-a114-a58bd32e1315 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:06.813822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb735578-3bb5-4a61-a114-a58bd32e1315 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:06.830654 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a89f61-8e55-4a30-b32d-b7f2a198d2bd None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:06.909607 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65444592-1213-49b8-911b-f8bf34b8abfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:06.910735 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65444592-1213-49b8-911b-f8bf34b8abfe None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:06.911213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65444592-1213-49b8-911b-f8bf34b8abfe None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:06.911727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65444592-1213-49b8-911b-f8bf34b8abfe None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:06.912625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65444592-1213-49b8-911b-f8bf34b8abfe None admin] RBAC: Adding query filter params (name=member) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:06.913129 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65444592-1213-49b8-911b-f8bf34b8abfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:06.914783 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65444592-1213-49b8-911b-f8bf34b8abfe None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:06.931196 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 84/167] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:06.940318 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65444592-1213-49b8-911b-f8bf34b8abfe None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:06.997602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:07.028035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 84/168] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:07.037579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb735578-3bb5-4a61-a114-a58bd32e1315 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:07.088273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:07.123437 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65444592-1213-49b8-911b-f8bf34b8abfe None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:07.196994 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3071b00-5812-4d1a-ab02-60a5c785ec85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:07.197644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3071b00-5812-4d1a-ab02-60a5c785ec85 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:07.197913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3071b00-5812-4d1a-ab02-60a5c785ec85 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:07.198178 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3071b00-5812-4d1a-ab02-60a5c785ec85 None admin] PATH_INFO: `/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:07.201985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3071b00-5812-4d1a-ab02-60a5c785ec85 None admin] RBAC: Authorizing `identity:get_user(user_id=8ee93dd82ab0424e9e7c9bdcc0d19a89)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:07.203452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3071b00-5812-4d1a-ab02-60a5c785ec85 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:07.206288 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 85/169] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:06 2022] GET /identity/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:07.207322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3071b00-5812-4d1a-ab02-60a5c785ec85 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:07.232988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb735578-3bb5-4a61-a114-a58bd32e1315 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:07.296035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6abf84-2c59-4d1a-9db9-c082bed9058e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:07.296678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6abf84-2c59-4d1a-9db9-c082bed9058e None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:07.296947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6abf84-2c59-4d1a-9db9-c082bed9058e None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:07.297210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6abf84-2c59-4d1a-9db9-c082bed9058e None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:07.298357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6abf84-2c59-4d1a-9db9-c082bed9058e None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:07.300061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6abf84-2c59-4d1a-9db9-c082bed9058e None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:07.301760 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 85/170] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:07 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:07.302871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bc4e6-d438-4a5f-8fe6-56a77ab41563 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:07.303148 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bc4e6-d438-4a5f-8fe6-56a77ab41563 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:07.303411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6bc4e6-d438-4a5f-8fe6-56a77ab41563 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:07.304627 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 86/171] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:07.305324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bc4e6-d438-4a5f-8fe6-56a77ab41563 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:07.396367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3071b00-5812-4d1a-ab02-60a5c785ec85 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:07.417678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:07.441571 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378a539-8668-4194-b8f3-572fcb0e421b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:07.442213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378a539-8668-4194-b8f3-572fcb0e421b None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:07.442485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378a539-8668-4194-b8f3-572fcb0e421b None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:07.442753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378a539-8668-4194-b8f3-572fcb0e421b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:07.461047 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378a539-8668-4194-b8f3-572fcb0e421b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-6378a539-8668-4194-b8f3-572fcb0e421b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:07.467526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:07.468422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:07.469690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:07.469690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:07.469690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:07.469690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:07.469690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:07.469690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:07.469690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:07.469690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:07.469690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:07.469690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:07.469690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:07.469690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:07.469690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 86/172] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:54:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:07.470368 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378a539-8668-4194-b8f3-572fcb0e421b None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:07.483533 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:07.521366 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6bc4e6-d438-4a5f-8fe6-56a77ab41563 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:07.591114 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3d8de-5bcb-4ad9-9f02-cbd9a4882ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:07.591764 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3d8de-5bcb-4ad9-9f02-cbd9a4882ade None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:07.592034 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3d8de-5bcb-4ad9-9f02-cbd9a4882ade None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:07.592305 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3d8de-5bcb-4ad9-9f02-cbd9a4882ade None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:07.647271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378a539-8668-4194-b8f3-572fcb0e421b None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:07.717504 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412ccec-f9c3-4ef4-b487-ff668958672f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:07.718122 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412ccec-f9c3-4ef4-b487-ff668958672f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:07.718395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412ccec-f9c3-4ef4-b487-ff668958672f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:07.718658 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412ccec-f9c3-4ef4-b487-ff668958672f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:07.719387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412ccec-f9c3-4ef4-b487-ff668958672f None admin] RBAC: Adding query filter params (role.id=abbd88ef58a544db8ee4261548778980, scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e, user.id=8ee93dd82ab0424e9e7c9bdcc0d19a89) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:07.719742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412ccec-f9c3-4ef4-b487-ff668958672f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:07.721261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412ccec-f9c3-4ef4-b487-ff668958672f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:07.722378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3d8de-5bcb-4ad9-9f02-cbd9a4882ade None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:07.723874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3d8de-5bcb-4ad9-9f02-cbd9a4882ade None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:07.733697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 87/173] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:07 2022] GET /identity/v3/role_assignments?user.id=8ee93dd82ab0424e9e7c9bdcc0d19a89&scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e&role.id=abbd88ef58a544db8ee4261548778980 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:07.734982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914eeeb5-7a3d-46fb-bd9d-a8cbf86406b7 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:07.735264 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914eeeb5-7a3d-46fb-bd9d-a8cbf86406b7 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:07.741941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914eeeb5-7a3d-46fb-bd9d-a8cbf86406b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:07.781854 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:07.783354 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-914eeeb5-7a3d-46fb-bd9d-a8cbf86406b7 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:07.784597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-914eeeb5-7a3d-46fb-bd9d-a8cbf86406b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:07.830148 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 87/174] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:07.831128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3d8de-5bcb-4ad9-9f02-cbd9a4882ade None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:07.945564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 88/175] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:07.946426 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914eeeb5-7a3d-46fb-bd9d-a8cbf86406b7 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:07.955176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:07.971436 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3d8de-5bcb-4ad9-9f02-cbd9a4882ade None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:08.008434 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcde99-358e-4647-a1f9-68c48cae794b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:08.009076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcde99-358e-4647-a1f9-68c48cae794b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:08.009332 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcde99-358e-4647-a1f9-68c48cae794b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:08.009607 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcde99-358e-4647-a1f9-68c48cae794b None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:08.010299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdcde99-358e-4647-a1f9-68c48cae794b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:08.010602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdcde99-358e-4647-a1f9-68c48cae794b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:08.012087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdcde99-358e-4647-a1f9-68c48cae794b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:08.021370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 88/176] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:08.022675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e747fd4-b892-48db-8ec4-2ad04fe3997e None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:08.022963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e747fd4-b892-48db-8ec4-2ad04fe3997e None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:08.023238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e747fd4-b892-48db-8ec4-2ad04fe3997e None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:08.024603 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 89/177] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:08.027708 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e747fd4-b892-48db-8ec4-2ad04fe3997e None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:08.049132 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:08.136499 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914eeeb5-7a3d-46fb-bd9d-a8cbf86406b7 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:08.152819 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e747fd4-b892-48db-8ec4-2ad04fe3997e None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:08.196670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507ce7e-4946-41bb-a724-0e29962870d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:08.197249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507ce7e-4946-41bb-a724-0e29962870d3 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:08.197573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507ce7e-4946-41bb-a724-0e29962870d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:08.197837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507ce7e-4946-41bb-a724-0e29962870d3 None admin] PATH_INFO: `/v3/projects/782b3b19a4f54723abdbb3b3cf184ef4` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:08.200022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507ce7e-4946-41bb-a724-0e29962870d3 None admin] RBAC: Authorizing `identity:get_project(project_id=782b3b19a4f54723abdbb3b3cf184ef4)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:08.201496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507ce7e-4946-41bb-a724-0e29962870d3 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:08.203349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 89/178] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:07 2022] GET /identity/v3/projects/782b3b19a4f54723abdbb3b3cf184ef4 => generated 312 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:08.204259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e2375-c52c-4e98-b496-8d1112a1a47c None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:08.204533 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e2375-c52c-4e98-b496-8d1112a1a47c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:08.204857 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967e2375-c52c-4e98-b496-8d1112a1a47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:08.218378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606aebda-cfd1-4884-82b7-40af95b85139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:08.219098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606aebda-cfd1-4884-82b7-40af95b85139 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:08.219664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606aebda-cfd1-4884-82b7-40af95b85139 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:08.219937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606aebda-cfd1-4884-82b7-40af95b85139 None admin] PATH_INFO: `/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:08.221117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606aebda-cfd1-4884-82b7-40af95b85139 None admin] RBAC: Authorizing `identity:get_user(user_id=8ee93dd82ab0424e9e7c9bdcc0d19a89)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:08.222733 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606aebda-cfd1-4884-82b7-40af95b85139 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:08.223507 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-967e2375-c52c-4e98-b496-8d1112a1a47c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:08.224770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 90/179] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:08 2022] GET /identity/v3/users/8ee93dd82ab0424e9e7c9bdcc0d19a89 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:08.224879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-967e2375-c52c-4e98-b496-8d1112a1a47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:08.225546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606aebda-cfd1-4884-82b7-40af95b85139 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:08.336819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 90/180] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:08.337677 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e2375-c52c-4e98-b496-8d1112a1a47c None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:08.342345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:08.384539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:08.422003 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606aebda-cfd1-4884-82b7-40af95b85139 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:08.479121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967e2375-c52c-4e98-b496-8d1112a1a47c None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:08.492707 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b14ca-7cef-4615-b63c-0c52534d4b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:08.493341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b14ca-7cef-4615-b63c-0c52534d4b66 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:08.493663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b14ca-7cef-4615-b63c-0c52534d4b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:08.493943 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b14ca-7cef-4615-b63c-0c52534d4b66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:08.494676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b14ca-7cef-4615-b63c-0c52534d4b66 None admin] RBAC: Adding query filter params (role.id=abbd88ef58a544db8ee4261548778980, scope.project.id=782b3b19a4f54723abdbb3b3cf184ef4, user.id=8ee93dd82ab0424e9e7c9bdcc0d19a89) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:08.494987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b14ca-7cef-4615-b63c-0c52534d4b66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:08.496458 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b14ca-7cef-4615-b63c-0c52534d4b66 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:08.509798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 91/181] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:08 2022] GET /identity/v3/role_assignments?user.id=8ee93dd82ab0424e9e7c9bdcc0d19a89&scope.project.id=782b3b19a4f54723abdbb3b3cf184ef4&role.id=abbd88ef58a544db8ee4261548778980 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:08.510980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b14ca-7cef-4615-b63c-0c52534d4b66 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:08.521014 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e39d99-3317-4b26-8c1a-bb6b933af3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:08.521657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e39d99-3317-4b26-8c1a-bb6b933af3a8 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:08.521930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e39d99-3317-4b26-8c1a-bb6b933af3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:08.522195 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e39d99-3317-4b26-8c1a-bb6b933af3a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:08.661428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e39d99-3317-4b26-8c1a-bb6b933af3a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:08.662922 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e39d99-3317-4b26-8c1a-bb6b933af3a8 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:08.665343 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:08.694637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b14ca-7cef-4615-b63c-0c52534d4b66 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:08.739517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5cb7a0-e248-4b18-9f85-e07a4c4e5984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:08.740201 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5cb7a0-e248-4b18-9f85-e07a4c4e5984 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:08.740479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5cb7a0-e248-4b18-9f85-e07a4c4e5984 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:08.740744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5cb7a0-e248-4b18-9f85-e07a4c4e5984 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:08.752220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5cb7a0-e248-4b18-9f85-e07a4c4e5984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-df5cb7a0-e248-4b18-9f85-e07a4c4e5984 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:08.758027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:08.759082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:08.760341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:08.760341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:08.760341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:08.760341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:08.760341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:08.760341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:08.760341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:08.760341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:08.760341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:08.760341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:08.760341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:08.760341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:08.760341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 92/182] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:08.761017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d13d4-f32f-4a92-9d83-dc48040326ab None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:08.761017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d13d4-f32f-4a92-9d83-dc48040326ab None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:08.761017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d13d4-f32f-4a92-9d83-dc48040326ab None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:08.766413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 93/183] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:08.766413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d13d4-f32f-4a92-9d83-dc48040326ab None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:08.776713 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:08.818970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 91/184] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:08.820251 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e39d99-3317-4b26-8c1a-bb6b933af3a8 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:08.895349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d13d4-f32f-4a92-9d83-dc48040326ab None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:08.925545 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e39d99-3317-4b26-8c1a-bb6b933af3a8 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:08.927050 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:08.927922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72addf19-a0f3-44c6-b38a-41c17e80d697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:08.928560 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72addf19-a0f3-44c6-b38a-41c17e80d697 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:08.928861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72addf19-a0f3-44c6-b38a-41c17e80d697 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:08.929130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72addf19-a0f3-44c6-b38a-41c17e80d697 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:08.940301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72addf19-a0f3-44c6-b38a-41c17e80d697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-72addf19-a0f3-44c6-b38a-41c17e80d697 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:08.945682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:08.946741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:08.948155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:08.948155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:08.948155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:08.948155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:08.948155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:08.948155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:08.948155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:08.948155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:08.948155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:08.948155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:08.948155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:54:08.948155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:08.948155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 94/185] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:54:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:08.948155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2d63d-52c7-436a-b8a2-efca0ac1932b None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:08.948875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2d63d-52c7-436a-b8a2-efca0ac1932b None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:08.948875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2d63d-52c7-436a-b8a2-efca0ac1932b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:08.965232 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418626a-f4a4-4830-9f94-7dca7af1b29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:08.965851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418626a-f4a4-4830-9f94-7dca7af1b29f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:08.966143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418626a-f4a4-4830-9f94-7dca7af1b29f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:08.966261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-6dc2d63d-52c7-436a-b8a2-efca0ac1932b None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:08.966395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418626a-f4a4-4830-9f94-7dca7af1b29f None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:08.967064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418626a-f4a4-4830-9f94-7dca7af1b29f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:08.967361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418626a-f4a4-4830-9f94-7dca7af1b29f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:08.967471 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6dc2d63d-52c7-436a-b8a2-efca0ac1932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:08.968782 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418626a-f4a4-4830-9f94-7dca7af1b29f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:08.977051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 92/186] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:08.977819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418626a-f4a4-4830-9f94-7dca7af1b29f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:09.030735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:09.045431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 95/187] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:09.045431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f245f-5d60-4279-8a8c-2b852ed02d35 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:09.045431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f245f-5d60-4279-8a8c-2b852ed02d35 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:09.045431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f245f-5d60-4279-8a8c-2b852ed02d35 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:09.045431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 96/188] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:09.045431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f245f-5d60-4279-8a8c-2b852ed02d35 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:09.102458 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418626a-f4a4-4830-9f94-7dca7af1b29f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:09.135044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ac23d-74a3-448d-a32d-baa0a776a052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:09.135711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ac23d-74a3-448d-a32d-baa0a776a052 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:09.135981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ac23d-74a3-448d-a32d-baa0a776a052 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:09.136274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ac23d-74a3-448d-a32d-baa0a776a052 None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:09.137393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ac23d-74a3-448d-a32d-baa0a776a052 None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:09.138862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671ac23d-74a3-448d-a32d-baa0a776a052 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:09.140940 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 93/189] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:08 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:09.141702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ac23d-74a3-448d-a32d-baa0a776a052 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:09.166313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f245f-5d60-4279-8a8c-2b852ed02d35 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:09.175953 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:09.208367 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f459d2e-a8d3-47c4-b241-e395bce1ece1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:09.208989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f459d2e-a8d3-47c4-b241-e395bce1ece1 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:09.209265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f459d2e-a8d3-47c4-b241-e395bce1ece1 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:09.209536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f459d2e-a8d3-47c4-b241-e395bce1ece1 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:09.210206 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f459d2e-a8d3-47c4-b241-e395bce1ece1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:09.210515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f459d2e-a8d3-47c4-b241-e395bce1ece1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:09.211949 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f459d2e-a8d3-47c4-b241-e395bce1ece1 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:09.220578 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 97/190] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:09.222600 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9e3f4-e911-4f3a-95c0-819fcaec5258 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:09.222901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9e3f4-e911-4f3a-95c0-819fcaec5258 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:09.223173 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9e3f4-e911-4f3a-95c0-819fcaec5258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:09.243008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-d6c9e3f4-e911-4f3a-95c0-819fcaec5258 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:09.244229 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d6c9e3f4-e911-4f3a-95c0-819fcaec5258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:09.256725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ac23d-74a3-448d-a32d-baa0a776a052 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:09.298219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237eb08d-05e2-4574-a2a1-3c15c1b1a742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:09.299469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237eb08d-05e2-4574-a2a1-3c15c1b1a742 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:09.300154 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237eb08d-05e2-4574-a2a1-3c15c1b1a742 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:09.300686 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237eb08d-05e2-4574-a2a1-3c15c1b1a742 None admin] PATH_INFO: `/v3/users/5cd6191586a2482f8a9120be284d5761` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:09.302000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:09.302627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237eb08d-05e2-4574-a2a1-3c15c1b1a742 None admin] RBAC: Authorizing `identity:get_user(user_id=5cd6191586a2482f8a9120be284d5761)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:09.304283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237eb08d-05e2-4574-a2a1-3c15c1b1a742 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:09.310345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 94/191] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:09 2022] GET /identity/v3/users/5cd6191586a2482f8a9120be284d5761 => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:09.310345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237eb08d-05e2-4574-a2a1-3c15c1b1a742 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:09.313609 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 98/192] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:09.314444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9e3f4-e911-4f3a-95c0-819fcaec5258 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:09.409296 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:09.413139 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237eb08d-05e2-4574-a2a1-3c15c1b1a742 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:09.415203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9e3f4-e911-4f3a-95c0-819fcaec5258 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:09.448115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbe291-b6d0-4ce9-8d09-5e78140fe6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:09.448722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbe291-b6d0-4ce9-8d09-5e78140fe6ca None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:09.448995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbe291-b6d0-4ce9-8d09-5e78140fe6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:09.449265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebbe291-b6d0-4ce9-8d09-5e78140fe6ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:09.449986 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbe291-b6d0-4ce9-8d09-5e78140fe6ca None admin] RBAC: Adding query filter params (role.id=bf98c97818504bc48e3bd1b2aef9d3c3, scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e, user.id=8ee93dd82ab0424e9e7c9bdcc0d19a89) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:09.450315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbe291-b6d0-4ce9-8d09-5e78140fe6ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:09.451348 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40ad96-47fa-4811-a6f1-9273d597f865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:09.451784 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebbe291-b6d0-4ce9-8d09-5e78140fe6ca None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:09.452032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40ad96-47fa-4811-a6f1-9273d597f865 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:09.452306 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40ad96-47fa-4811-a6f1-9273d597f865 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:09.452571 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40ad96-47fa-4811-a6f1-9273d597f865 None admin] PATH_INFO: `/v3/users/8f442bafd9f1464f9d625ffec27c57e7` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:09.467873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 95/193] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:09 2022] GET /identity/v3/role_assignments?user.id=8ee93dd82ab0424e9e7c9bdcc0d19a89&scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e&role.id=bf98c97818504bc48e3bd1b2aef9d3c3 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:09.467873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbe291-b6d0-4ce9-8d09-5e78140fe6ca None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:09.476793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40ad96-47fa-4811-a6f1-9273d597f865 None admin] RBAC: Authorizing `identity:get_user(user_id=8f442bafd9f1464f9d625ffec27c57e7)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:09.478437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a40ad96-47fa-4811-a6f1-9273d597f865 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:09.480328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 99/194] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:09 2022] GET /identity/v3/users/8f442bafd9f1464f9d625ffec27c57e7 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:09.481218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40ad96-47fa-4811-a6f1-9273d597f865 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:09.536591 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:09.579851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebbe291-b6d0-4ce9-8d09-5e78140fe6ca None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:09.594963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a40ad96-47fa-4811-a6f1-9273d597f865 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:09.618340 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e2596-9f2c-4f15-b9bd-055c50927015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:09.618999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e2596-9f2c-4f15-b9bd-055c50927015 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:09.619355 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e2596-9f2c-4f15-b9bd-055c50927015 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:09.619586 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e2596-9f2c-4f15-b9bd-055c50927015 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:09.631315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e2596-9f2c-4f15-b9bd-055c50927015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:09.631857 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d080dd2-ef51-4a15-9a17-ee89de04db33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:09.632504 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d080dd2-ef51-4a15-9a17-ee89de04db33 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:09.632777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d080dd2-ef51-4a15-9a17-ee89de04db33 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:09.633043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d080dd2-ef51-4a15-9a17-ee89de04db33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-0d2e2596-9f2c-4f15-b9bd-055c50927015 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:09.637758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:09.638751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:09.639602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:09.639602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:09.639602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:09.639602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:09.639602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:09.639602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:09.639602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:09.639602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:09.639602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:09.639602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:09.639602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:09.639602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:09.639602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 96/195] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:54:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:09.639602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e2596-9f2c-4f15-b9bd-055c50927015 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:09.668388 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:09.715652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d080dd2-ef51-4a15-9a17-ee89de04db33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:09.717140 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d080dd2-ef51-4a15-9a17-ee89de04db33 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:09.752184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e2596-9f2c-4f15-b9bd-055c50927015 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:09.769152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:09.786112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67602b8e-10e5-49fe-a853-d3459cfd0a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:09.786723 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67602b8e-10e5-49fe-a853-d3459cfd0a1f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:09.786998 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67602b8e-10e5-49fe-a853-d3459cfd0a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:09.787267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67602b8e-10e5-49fe-a853-d3459cfd0a1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:09.819249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 100/196] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:09.820127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d080dd2-ef51-4a15-9a17-ee89de04db33 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:09.854193 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67602b8e-10e5-49fe-a853-d3459cfd0a1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:09.855621 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67602b8e-10e5-49fe-a853-d3459cfd0a1f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:09.879416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:09.917582 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d080dd2-ef51-4a15-9a17-ee89de04db33 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:09.951391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a9530-7f4c-4d3f-b373-68c8f7a5c442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:09.952054 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a9530-7f4c-4d3f-b373-68c8f7a5c442 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:09.952325 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a9530-7f4c-4d3f-b373-68c8f7a5c442 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:09.952591 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a9530-7f4c-4d3f-b373-68c8f7a5c442 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:09.952998 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 97/197] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:09.957822 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67602b8e-10e5-49fe-a853-d3459cfd0a1f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:09.958738 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a9530-7f4c-4d3f-b373-68c8f7a5c442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:09.960318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a9530-7f4c-4d3f-b373-68c8f7a5c442 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:09.961920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 101/198] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:09.962691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a9530-7f4c-4d3f-b373-68c8f7a5c442 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:09.981349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:10.051887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67602b8e-10e5-49fe-a853-d3459cfd0a1f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:10.062825 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a9530-7f4c-4d3f-b373-68c8f7a5c442 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:10.074512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:10.085248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07b6f5-9fab-4a8d-a5a6-b37ee3858e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:10.085864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07b6f5-9fab-4a8d-a5a6-b37ee3858e45 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:10.086134 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07b6f5-9fab-4a8d-a5a6-b37ee3858e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:10.086401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07b6f5-9fab-4a8d-a5a6-b37ee3858e45 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:10.087068 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07b6f5-9fab-4a8d-a5a6-b37ee3858e45 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:10.087366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07b6f5-9fab-4a8d-a5a6-b37ee3858e45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:10.089192 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07b6f5-9fab-4a8d-a5a6-b37ee3858e45 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:10.096549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e4518-152d-4cd8-9b58-21681bedbaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:10.096898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 98/199] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:10.097710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07b6f5-9fab-4a8d-a5a6-b37ee3858e45 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:10.099796 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e4518-152d-4cd8-9b58-21681bedbaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:10.100193 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e4518-152d-4cd8-9b58-21681bedbaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:10.100471 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e4518-152d-4cd8-9b58-21681bedbaa7 None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:10.101790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e4518-152d-4cd8-9b58-21681bedbaa7 None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:10.103244 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e4518-152d-4cd8-9b58-21681bedbaa7 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:10.105012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 102/200] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:09 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:10.105704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e4518-152d-4cd8-9b58-21681bedbaa7 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:10.188852 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:10.199464 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07b6f5-9fab-4a8d-a5a6-b37ee3858e45 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:10.204436 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e4518-152d-4cd8-9b58-21681bedbaa7 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:10.232536 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d5c43-dc6f-40df-822f-0bf32b72021e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:10.233165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d5c43-dc6f-40df-822f-0bf32b72021e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:10.233436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d5c43-dc6f-40df-822f-0bf32b72021e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:10.233699 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d5c43-dc6f-40df-822f-0bf32b72021e None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:10.234858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d5c43-dc6f-40df-822f-0bf32b72021e None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:10.236349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3d5c43-dc6f-40df-822f-0bf32b72021e None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:10.237977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 99/201] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:10 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:10.238729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d5c43-dc6f-40df-822f-0bf32b72021e None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:10.239150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407346c-74ab-44d7-8572-11f834891e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:10.239804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407346c-74ab-44d7-8572-11f834891e14 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:10.240081 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407346c-74ab-44d7-8572-11f834891e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:10.240348 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8407346c-74ab-44d7-8572-11f834891e14 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:10.241120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407346c-74ab-44d7-8572-11f834891e14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:10.242931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8407346c-74ab-44d7-8572-11f834891e14 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:10.263891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 103/202] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:54:10 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:54:10.264808 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407346c-74ab-44d7-8572-11f834891e14 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:10.296274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:10.351299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3d5c43-dc6f-40df-822f-0bf32b72021e None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:10.390274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc15aec-e79c-42f2-8d91-41c1f218fa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:10.391205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc15aec-e79c-42f2-8d91-41c1f218fa4a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:10.391652 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc15aec-e79c-42f2-8d91-41c1f218fa4a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:10.392081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc15aec-e79c-42f2-8d91-41c1f218fa4a None admin] PATH_INFO: `/v3/users/8f442bafd9f1464f9d625ffec27c57e7` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:10.393321 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc15aec-e79c-42f2-8d91-41c1f218fa4a None admin] RBAC: Authorizing `identity:get_user(user_id=8f442bafd9f1464f9d625ffec27c57e7)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:10.394896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc15aec-e79c-42f2-8d91-41c1f218fa4a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:10.396026 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8407346c-74ab-44d7-8572-11f834891e14 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:10.401388 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 100/203] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:10 2022] GET /identity/v3/users/8f442bafd9f1464f9d625ffec27c57e7 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:10.401388 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc15aec-e79c-42f2-8d91-41c1f218fa4a None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:10.439059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5bcb0-853a-4b52-a371-e4f8ac1d2f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:10.439704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5bcb0-853a-4b52-a371-e4f8ac1d2f47 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:10.440651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5bcb0-853a-4b52-a371-e4f8ac1d2f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:10.440651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e5bcb0-853a-4b52-a371-e4f8ac1d2f47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:10.441073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5bcb0-853a-4b52-a371-e4f8ac1d2f47 None admin] RBAC: Adding query filter params (role.id=a9c50f8878284257bb064b71a6f7870b, scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e, user.id=5cd6191586a2482f8a9120be284d5761) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:10.441837 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5bcb0-853a-4b52-a371-e4f8ac1d2f47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:10.442825 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e5bcb0-853a-4b52-a371-e4f8ac1d2f47 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:10.449414 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 104/204] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:10 2022] GET /identity/v3/role_assignments?user.id=5cd6191586a2482f8a9120be284d5761&scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e&role.id=a9c50f8878284257bb064b71a6f7870b => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:10.449875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5bcb0-853a-4b52-a371-e4f8ac1d2f47 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:10.460800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:10.523971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc15aec-e79c-42f2-8d91-41c1f218fa4a None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:10.569306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:10.584500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2a143-1920-45c9-b4fa-ee6a553b284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:10.586682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2a143-1920-45c9-b4fa-ee6a553b284d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:10.587617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2a143-1920-45c9-b4fa-ee6a553b284d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:10.588721 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2a143-1920-45c9-b4fa-ee6a553b284d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:10.589763 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2a143-1920-45c9-b4fa-ee6a553b284d None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd, user.id=8f442bafd9f1464f9d625ffec27c57e7) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:10.590079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2a143-1920-45c9-b4fa-ee6a553b284d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:10.590431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e5bcb0-853a-4b52-a371-e4f8ac1d2f47 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:10.591608 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2a143-1920-45c9-b4fa-ee6a553b284d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:10.599900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 101/205] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:10 2022] GET /identity/v3/role_assignments?user.id=8f442bafd9f1464f9d625ffec27c57e7&scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:10.639540 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe84d9-42dc-41e9-a20c-2e1fa1b4bfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:10.640533 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe84d9-42dc-41e9-a20c-2e1fa1b4bfce None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:10.640975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe84d9-42dc-41e9-a20c-2e1fa1b4bfce None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:10.641435 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe84d9-42dc-41e9-a20c-2e1fa1b4bfce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:10.727659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe84d9-42dc-41e9-a20c-2e1fa1b4bfce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:10.729429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe84d9-42dc-41e9-a20c-2e1fa1b4bfce None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:10.738470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:10.841514 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 105/206] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:10.847370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2a143-1920-45c9-b4fa-ee6a553b284d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:10.954143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2a143-1920-45c9-b4fa-ee6a553b284d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:10.991327 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1609eb-2f8e-4589-84b9-5801772d3dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:10.992405 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1609eb-2f8e-4589-84b9-5801772d3dab None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:10.992872 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1609eb-2f8e-4589-84b9-5801772d3dab None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:10.993363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1609eb-2f8e-4589-84b9-5801772d3dab None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:10.994742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1609eb-2f8e-4589-84b9-5801772d3dab None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:10.996436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1609eb-2f8e-4589-84b9-5801772d3dab None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:10.998790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 102/207] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:10 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:11.005312 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe84d9-42dc-41e9-a20c-2e1fa1b4bfce None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:11.109848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe84d9-42dc-41e9-a20c-2e1fa1b4bfce None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:11.117498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:11.144429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0430979-4441-4bd1-966a-5222a90f513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:11.145048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0430979-4441-4bd1-966a-5222a90f513c None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:11.145323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0430979-4441-4bd1-966a-5222a90f513c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:11.145593 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0430979-4441-4bd1-966a-5222a90f513c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:11.146327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0430979-4441-4bd1-966a-5222a90f513c None admin] RBAC: Adding query filter params (role.id=bf98c97818504bc48e3bd1b2aef9d3c3, scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd, user.id=8f442bafd9f1464f9d625ffec27c57e7) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:11.146629 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0430979-4441-4bd1-966a-5222a90f513c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:11.148113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0430979-4441-4bd1-966a-5222a90f513c None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:11.153577 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 106/208] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:11 2022] GET /identity/v3/role_assignments?user.id=8f442bafd9f1464f9d625ffec27c57e7&scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd&role.id=bf98c97818504bc48e3bd1b2aef9d3c3 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:12.747731 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd1ca0-1545-443f-80ea-0abc0c907b47 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:12.748735 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd1ca0-1545-443f-80ea-0abc0c907b47 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:12.749238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd1ca0-1545-443f-80ea-0abc0c907b47 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:12.751044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 103/209] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:12.757471 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c4555-cf40-4c0c-9923-430eea480312 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:12.758427 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c4555-cf40-4c0c-9923-430eea480312 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:12.758913 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c4555-cf40-4c0c-9923-430eea480312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:12.779551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-613c4555-cf40-4c0c-9923-430eea480312 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:12.781105 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-613c4555-cf40-4c0c-9923-430eea480312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:12.806040 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095f4aa-ff59-498b-a303-5903d8bc11df None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:12.806665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095f4aa-ff59-498b-a303-5903d8bc11df None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:12.807131 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095f4aa-ff59-498b-a303-5903d8bc11df None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:12.809032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 104/210] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:12.819929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618c906-f94a-45a7-a0bc-6ce9420af404 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:12.820469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618c906-f94a-45a7-a0bc-6ce9420af404 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:12.820938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618c906-f94a-45a7-a0bc-6ce9420af404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:12.839837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-e618c906-f94a-45a7-a0bc-6ce9420af404 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:12.841363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e618c906-f94a-45a7-a0bc-6ce9420af404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:12.855921 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 107/211] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:12.896659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c706b6-316a-408c-b2f3-c9a71d81811e None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:12.897310 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c706b6-316a-408c-b2f3-c9a71d81811e None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:12.897885 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:12.899050 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c706b6-316a-408c-b2f3-c9a71d81811e None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:12.901011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 108/212] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:54:12.908550 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d9e8f-5d3f-43a5-9a2c-a1d1d49c1585 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:12.909103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d9e8f-5d3f-43a5-9a2c-a1d1d49c1585 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:12.909595 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d9e8f-5d3f-43a5-9a2c-a1d1d49c1585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:12.927528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 105/213] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:12.936045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-887d9e8f-5d3f-43a5-9a2c-a1d1d49c1585 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:12.937901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-887d9e8f-5d3f-43a5-9a2c-a1d1d49c1585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:13.013262 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 109/214] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:13.130040 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618c906-f94a-45a7-a0bc-6ce9420af404 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:13.228286 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d9e8f-5d3f-43a5-9a2c-a1d1d49c1585 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:13.241515 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618c906-f94a-45a7-a0bc-6ce9420af404 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:13.260142 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:13.280107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef20ed4-7556-4e19-9915-364ff9b271c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:13.280982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef20ed4-7556-4e19-9915-364ff9b271c8 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:13.281474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef20ed4-7556-4e19-9915-364ff9b271c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:13.281876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef20ed4-7556-4e19-9915-364ff9b271c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:13.293270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef20ed4-7556-4e19-9915-364ff9b271c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-2ef20ed4-7556-4e19-9915-364ff9b271c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:13.299145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:13.301019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:13.302488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:13.302488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:13.302488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:13.302488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:13.302488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:13.302488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:13.302488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:13.302488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:13.302488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:13.302488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:13.302488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:13.302488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:13.302488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 106/215] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:13.306352 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8531c-d18b-40ad-ac96-58e1a14bc382 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:13.306872 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8531c-d18b-40ad-ac96-58e1a14bc382 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:13.307143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8531c-d18b-40ad-ac96-58e1a14bc382 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:13.308532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 107/216] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:13.309235 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8531c-d18b-40ad-ac96-58e1a14bc382 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:13.357083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:13.359130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d9e8f-5d3f-43a5-9a2c-a1d1d49c1585 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:13.402039 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd813c8-a13b-4a43-95cd-9a6db3ab50d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:13.402688 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd813c8-a13b-4a43-95cd-9a6db3ab50d8 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:13.402990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd813c8-a13b-4a43-95cd-9a6db3ab50d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:13.403258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd813c8-a13b-4a43-95cd-9a6db3ab50d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:13.404408 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd813c8-a13b-4a43-95cd-9a6db3ab50d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:13.405868 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd813c8-a13b-4a43-95cd-9a6db3ab50d8 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:13.407485 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 110/217] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:13.408269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd813c8-a13b-4a43-95cd-9a6db3ab50d8 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:13.420073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8531c-d18b-40ad-ac96-58e1a14bc382 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:13.453423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0215610-72e4-407d-9637-d6fb5625f744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:13.454059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0215610-72e4-407d-9637-d6fb5625f744 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:13.454320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0215610-72e4-407d-9637-d6fb5625f744 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:13.454592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0215610-72e4-407d-9637-d6fb5625f744 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:13.464854 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0215610-72e4-407d-9637-d6fb5625f744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-f0215610-72e4-407d-9637-d6fb5625f744 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:13.469753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:13.470687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:13.472684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:13.472684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:13.472684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:13.472684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:13.472684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:13.472684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:13.472684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:13.472684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:13.472684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:13.472684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:13.472684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:54:13.472684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:13.472684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 108/218] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:54:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:13.472684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84483b-42dc-4c8c-b59d-4da933defd62 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:13.473567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84483b-42dc-4c8c-b59d-4da933defd62 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:13.473567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84483b-42dc-4c8c-b59d-4da933defd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:13.490385 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-cd84483b-42dc-4c8c-b59d-4da933defd62 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:13.491661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-cd84483b-42dc-4c8c-b59d-4da933defd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:13.508234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:13.511999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd813c8-a13b-4a43-95cd-9a6db3ab50d8 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:13.546798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd35e44-098c-4279-a3ca-07bbe89a7880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:13.547432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd35e44-098c-4279-a3ca-07bbe89a7880 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:13.547772 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd35e44-098c-4279-a3ca-07bbe89a7880 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:13.548022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd35e44-098c-4279-a3ca-07bbe89a7880 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:13.548684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd35e44-098c-4279-a3ca-07bbe89a7880 None admin] RBAC: Adding query filter params (name=member) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:13.548979 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd35e44-098c-4279-a3ca-07bbe89a7880 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:13.550594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd35e44-098c-4279-a3ca-07bbe89a7880 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:13.559845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 111/219] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:13.560710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd35e44-098c-4279-a3ca-07bbe89a7880 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:13.563620 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 109/220] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:13.564474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84483b-42dc-4c8c-b59d-4da933defd62 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:13.605463 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:13.664433 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd35e44-098c-4279-a3ca-07bbe89a7880 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:13.673718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84483b-42dc-4c8c-b59d-4da933defd62 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:13.699816 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e36955-0567-415b-b8b2-096342d477e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:13.700436 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e36955-0567-415b-b8b2-096342d477e6 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:13.700698 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e36955-0567-415b-b8b2-096342d477e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:13.700965 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e36955-0567-415b-b8b2-096342d477e6 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:13.701614 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e36955-0567-415b-b8b2-096342d477e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:13.703142 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e36955-0567-415b-b8b2-096342d477e6 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:13.710187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb033ce-1e0f-417b-a3a0-af500d15ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:13.710820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb033ce-1e0f-417b-a3a0-af500d15ac4d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:13.711100 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb033ce-1e0f-417b-a3a0-af500d15ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:13.711378 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb033ce-1e0f-417b-a3a0-af500d15ac4d None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:13.712081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb033ce-1e0f-417b-a3a0-af500d15ac4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:13.712381 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb033ce-1e0f-417b-a3a0-af500d15ac4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:13.714148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb033ce-1e0f-417b-a3a0-af500d15ac4d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:13.724832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 110/221] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:13.725670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb033ce-1e0f-417b-a3a0-af500d15ac4d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:13.727823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 112/222] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:54:13 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:54:13.734465 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e36955-0567-415b-b8b2-096342d477e6 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:13.763797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:13.850825 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:13.858692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb033ce-1e0f-417b-a3a0-af500d15ac4d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:13.880840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e36955-0567-415b-b8b2-096342d477e6 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:13.897030 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ab80f-be4b-495f-b024-d7cb409f75cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:13.897677 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ab80f-be4b-495f-b024-d7cb409f75cb None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:13.897955 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ab80f-be4b-495f-b024-d7cb409f75cb None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:13.898221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ab80f-be4b-495f-b024-d7cb409f75cb None admin] PATH_INFO: `/v3/users/80e27756f7664a16a1d86c3e886e93c9` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:13.914722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ab80f-be4b-495f-b024-d7cb409f75cb None admin] RBAC: Authorizing `identity:get_user(user_id=80e27756f7664a16a1d86c3e886e93c9)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:13.916315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ab80f-be4b-495f-b024-d7cb409f75cb None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:13.919227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 111/223] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:13 2022] GET /identity/v3/users/80e27756f7664a16a1d86c3e886e93c9 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:13.919227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ab80f-be4b-495f-b024-d7cb409f75cb None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:13.925059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e5257-8e42-4950-95ae-736d98a46f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:13.925882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e5257-8e42-4950-95ae-736d98a46f4f None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:13.926166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e5257-8e42-4950-95ae-736d98a46f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:13.926395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e5257-8e42-4950-95ae-736d98a46f4f None admin] PATH_INFO: `/v3/users/8f442bafd9f1464f9d625ffec27c57e7` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:13.927609 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e5257-8e42-4950-95ae-736d98a46f4f None admin] RBAC: Authorizing `identity:get_user(user_id=8f442bafd9f1464f9d625ffec27c57e7)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:13.929040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e5257-8e42-4950-95ae-736d98a46f4f None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:13.930991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 113/224] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:13 2022] GET /identity/v3/users/8f442bafd9f1464f9d625ffec27c57e7 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:13.931869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e5257-8e42-4950-95ae-736d98a46f4f None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:14.033027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:14.045221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ab80f-be4b-495f-b024-d7cb409f75cb None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:14.056081 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e5257-8e42-4950-95ae-736d98a46f4f None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:14.088299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83682681-308c-41d8-841f-0fb77234475e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:14.088949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83682681-308c-41d8-841f-0fb77234475e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:14.089220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83682681-308c-41d8-841f-0fb77234475e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:14.089482 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83682681-308c-41d8-841f-0fb77234475e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:14.097248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6e048-b6a4-4fe5-af71-3137066a713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:14.098000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6e048-b6a4-4fe5-af71-3137066a713b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:14.098368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6e048-b6a4-4fe5-af71-3137066a713b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:14.098649 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6e048-b6a4-4fe5-af71-3137066a713b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:14.101615 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83682681-308c-41d8-841f-0fb77234475e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-83682681-308c-41d8-841f-0fb77234475e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:14.108044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:14.108929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:14.109711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:14.109711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:14.109711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:14.109711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:14.109711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:14.109711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:14.109711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:14.109711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:14.109711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:14.109711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:14.109711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:14.109711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:14.110302 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 112/225] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:54:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:14.111076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83682681-308c-41d8-841f-0fb77234475e None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:14.130459 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:14.188786 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6e048-b6a4-4fe5-af71-3137066a713b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:14.191474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6e048-b6a4-4fe5-af71-3137066a713b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:14.250740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83682681-308c-41d8-841f-0fb77234475e None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:14.285645 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:14.287531 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef8615-422b-4d61-8723-590ac80a6b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:14.288180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef8615-422b-4d61-8723-590ac80a6b1d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:14.288469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef8615-422b-4d61-8723-590ac80a6b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:14.288795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef8615-422b-4d61-8723-590ac80a6b1d None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:14.290089 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef8615-422b-4d61-8723-590ac80a6b1d None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:14.291584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef8615-422b-4d61-8723-590ac80a6b1d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:14.293446 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 113/226] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:14 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:14.298538 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef8615-422b-4d61-8723-590ac80a6b1d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:14.315578 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 114/227] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:14.317045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6e048-b6a4-4fe5-af71-3137066a713b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:14.387624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:14.412112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef8615-422b-4d61-8723-590ac80a6b1d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:14.435887 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6e048-b6a4-4fe5-af71-3137066a713b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:14.449378 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee115dc-4af7-4a24-8911-d4d1b232b105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:14.450232 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee115dc-4af7-4a24-8911-d4d1b232b105 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:14.450696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee115dc-4af7-4a24-8911-d4d1b232b105 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:14.451140 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee115dc-4af7-4a24-8911-d4d1b232b105 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:14.452085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee115dc-4af7-4a24-8911-d4d1b232b105 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:14.452600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee115dc-4af7-4a24-8911-d4d1b232b105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:14.454198 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee115dc-4af7-4a24-8911-d4d1b232b105 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:14.463300 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 114/228] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:14.468653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee115dc-4af7-4a24-8911-d4d1b232b105 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:14.473417 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b89db-ecbe-4e6e-9a6c-0a9bef907aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:14.473417 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b89db-ecbe-4e6e-9a6c-0a9bef907aaf None admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:14.473649 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b89db-ecbe-4e6e-9a6c-0a9bef907aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:14.473866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b89db-ecbe-4e6e-9a6c-0a9bef907aaf None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd/users/8f442bafd9f1464f9d625ffec27c57e7/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:14.475832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b89db-ecbe-4e6e-9a6c-0a9bef907aaf None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb5f153291f4fb28a4ccf24849d2ddd, user_id=8f442bafd9f1464f9d625ffec27c57e7, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:14.477275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b89db-ecbe-4e6e-9a6c-0a9bef907aaf None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:14.499476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 115/229] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:54:14 2022] PUT /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd/users/8f442bafd9f1464f9d625ffec27c57e7/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:54:14.499476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b89db-ecbe-4e6e-9a6c-0a9bef907aaf None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:14.555952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:14.615799 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b89db-ecbe-4e6e-9a6c-0a9bef907aaf None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:14.677124 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee115dc-4af7-4a24-8911-d4d1b232b105 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:14.678438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d93372-350b-486e-b6dc-92adc3a99ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:14.679217 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d93372-350b-486e-b6dc-92adc3a99ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:14.679518 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d93372-350b-486e-b6dc-92adc3a99ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:14.679787 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d93372-350b-486e-b6dc-92adc3a99ba7 None admin] PATH_INFO: `/v3/users/8f442bafd9f1464f9d625ffec27c57e7` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:14.681063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d93372-350b-486e-b6dc-92adc3a99ba7 None admin] RBAC: Authorizing `identity:get_user(user_id=8f442bafd9f1464f9d625ffec27c57e7)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:14.682640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d93372-350b-486e-b6dc-92adc3a99ba7 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:14.684388 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 116/230] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:14 2022] GET /identity/v3/users/8f442bafd9f1464f9d625ffec27c57e7 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:14.690339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d93372-350b-486e-b6dc-92adc3a99ba7 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:14.767649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1db05-9f6d-4b03-abff-73f0388bd710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:14.768701 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1db05-9f6d-4b03-abff-73f0388bd710 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:14.769209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1db05-9f6d-4b03-abff-73f0388bd710 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:14.769688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1db05-9f6d-4b03-abff-73f0388bd710 None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:14.771048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1db05-9f6d-4b03-abff-73f0388bd710 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:14.772774 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1db05-9f6d-4b03-abff-73f0388bd710 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:14.774963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 115/231] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:14 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:14.785147 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1db05-9f6d-4b03-abff-73f0388bd710 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:14.803224 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d93372-350b-486e-b6dc-92adc3a99ba7 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:14.819946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:14.832525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:14.845079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2560c-9b9b-4578-9a8a-592ae5a1315b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:14.845725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2560c-9b9b-4578-9a8a-592ae5a1315b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:14.845997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2560c-9b9b-4578-9a8a-592ae5a1315b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:14.846263 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e2560c-9b9b-4578-9a8a-592ae5a1315b None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:14.847595 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2560c-9b9b-4578-9a8a-592ae5a1315b None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:14.849111 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e2560c-9b9b-4578-9a8a-592ae5a1315b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:14.850905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 117/232] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:14 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:14.857111 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2560c-9b9b-4578-9a8a-592ae5a1315b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:14.965152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e2560c-9b9b-4578-9a8a-592ae5a1315b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:15.009040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f1376-8fb9-4e44-8505-883b19978fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:15.009715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f1376-8fb9-4e44-8505-883b19978fb1 None admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:15.009985 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f1376-8fb9-4e44-8505-883b19978fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:15.010250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f1376-8fb9-4e44-8505-883b19978fb1 None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd/users/8f442bafd9f1464f9d625ffec27c57e7/roles/bf98c97818504bc48e3bd1b2aef9d3c3` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:15.014638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f1376-8fb9-4e44-8505-883b19978fb1 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb5f153291f4fb28a4ccf24849d2ddd, user_id=8f442bafd9f1464f9d625ffec27c57e7, role_id=bf98c97818504bc48e3bd1b2aef9d3c3)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:15.016126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f1376-8fb9-4e44-8505-883b19978fb1 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:15.045698 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 118/233] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:54:14 2022] PUT /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd/users/8f442bafd9f1464f9d625ffec27c57e7/roles/bf98c97818504bc48e3bd1b2aef9d3c3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:54:15.066157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1db05-9f6d-4b03-abff-73f0388bd710 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:15.104466 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfb236-c92c-4b10-943d-b02002b78cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:15.105095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfb236-c92c-4b10-943d-b02002b78cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:15.105365 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfb236-c92c-4b10-943d-b02002b78cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:15.105686 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfb236-c92c-4b10-943d-b02002b78cb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:15.106517 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfb236-c92c-4b10-943d-b02002b78cb9 None admin] RBAC: Adding query filter params (role.id=abbd88ef58a544db8ee4261548778980, scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd, user.id=80e27756f7664a16a1d86c3e886e93c9) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:15.106819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfb236-c92c-4b10-943d-b02002b78cb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:15.108324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfb236-c92c-4b10-943d-b02002b78cb9 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:15.115196 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 116/234] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:14 2022] GET /identity/v3/role_assignments?user.id=80e27756f7664a16a1d86c3e886e93c9&scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd&role.id=abbd88ef58a544db8ee4261548778980 => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:16.220563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fcc19-8ddd-4c19-9221-48cb540c9916 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:16.221101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fcc19-8ddd-4c19-9221-48cb540c9916 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:16.221101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4fcc19-8ddd-4c19-9221-48cb540c9916 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:16.222548 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 119/235] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:16.231513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c6a56-5af4-4263-b42a-b6ac15f5caed None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:16.232039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c6a56-5af4-4263-b42a-b6ac15f5caed None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:16.232039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c6a56-5af4-4263-b42a-b6ac15f5caed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:16.257598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-743c6a56-5af4-4263-b42a-b6ac15f5caed None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:16.258836 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-743c6a56-5af4-4263-b42a-b6ac15f5caed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:16.343283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 117/236] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:16.355958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76588968-df31-4f98-8978-3196441c03b5 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:16.356231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76588968-df31-4f98-8978-3196441c03b5 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:16.356500 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76588968-df31-4f98-8978-3196441c03b5 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:16.357864 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 120/237] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:16.363516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3bc1e-c172-469f-b2d3-fba2446bd94d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:16.363808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3bc1e-c172-469f-b2d3-fba2446bd94d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:16.364067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3bc1e-c172-469f-b2d3-fba2446bd94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:16.377069 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:16.383374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-34b3bc1e-c172-469f-b2d3-fba2446bd94d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:16.384736 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-34b3bc1e-c172-469f-b2d3-fba2446bd94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:16.451127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 118/238] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:16.711284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76588968-df31-4f98-8978-3196441c03b5 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:16.733466 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3bc1e-c172-469f-b2d3-fba2446bd94d None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:16.760370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:16.843611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76588968-df31-4f98-8978-3196441c03b5 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:16.876655 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b3bc1e-c172-469f-b2d3-fba2446bd94d None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:16.905118 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299e3b1-b7f7-40a3-b351-52f12179d4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:16.905765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299e3b1-b7f7-40a3-b351-52f12179d4ca None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:16.906036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299e3b1-b7f7-40a3-b351-52f12179d4ca None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:16.906298 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299e3b1-b7f7-40a3-b351-52f12179d4ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:16.912163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299e3b1-b7f7-40a3-b351-52f12179d4ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:16.913638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4299e3b1-b7f7-40a3-b351-52f12179d4ca None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:16.915647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 121/239] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:16.921440 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299e3b1-b7f7-40a3-b351-52f12179d4ca None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:16.921629 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371e9b3-07a3-4050-b223-de263d606ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:16.922518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371e9b3-07a3-4050-b223-de263d606ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:16.922980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371e9b3-07a3-4050-b223-de263d606ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:16.923403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371e9b3-07a3-4050-b223-de263d606ff9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:16.926538 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:16.936367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371e9b3-07a3-4050-b223-de263d606ff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-5371e9b3-07a3-4050-b223-de263d606ff9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:16.941597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:16.942693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:16.943642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:16.943642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:16.943642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:16.943642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:16.943642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:16.943642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:16.943642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:16.943642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:16.943642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:16.943642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:16.943642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:16.943642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:16.945913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 119/240] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:54:16.951763 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371e9b3-07a3-4050-b223-de263d606ff9 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:17.057859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371e9b3-07a3-4050-b223-de263d606ff9 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:17.094156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25163e2-59af-4714-b77b-53a728ffc281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:17.095007 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25163e2-59af-4714-b77b-53a728ffc281 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:17.095606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25163e2-59af-4714-b77b-53a728ffc281 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:17.096018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25163e2-59af-4714-b77b-53a728ffc281 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:17.097109 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25163e2-59af-4714-b77b-53a728ffc281 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:17.097549 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25163e2-59af-4714-b77b-53a728ffc281 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:17.099160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25163e2-59af-4714-b77b-53a728ffc281 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:17.108757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 120/241] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:17.114338 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25163e2-59af-4714-b77b-53a728ffc281 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:17.212162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299e3b1-b7f7-40a3-b351-52f12179d4ca None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:17.222961 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:17.251810 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:17.264278 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab1a6d-d4fd-47b5-bde6-45011d477397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:17.265015 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab1a6d-d4fd-47b5-bde6-45011d477397 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:17.265295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab1a6d-d4fd-47b5-bde6-45011d477397 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:17.265563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab1a6d-d4fd-47b5-bde6-45011d477397 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:17.266235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab1a6d-d4fd-47b5-bde6-45011d477397 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:17.267834 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab1a6d-d4fd-47b5-bde6-45011d477397 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:17.301992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25163e2-59af-4714-b77b-53a728ffc281 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:17.311752 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 122/242] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:54:16 2022] POST /identity/v3/users => generated 293 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 15:54:17.313811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015fa1c-d866-4f2a-b9a0-c3bd4b830676 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:17.314599 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015fa1c-d866-4f2a-b9a0-c3bd4b830676 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:17.315117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9015fa1c-d866-4f2a-b9a0-c3bd4b830676 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:17.316897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 123/243] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:17.326202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ad847-ee7e-4325-abaf-0a04e66fcf26 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:17.326826 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ad847-ee7e-4325-abaf-0a04e66fcf26 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:17.327298 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ad847-ee7e-4325-abaf-0a04e66fcf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:17.351368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-3e0ad847-ee7e-4325-abaf-0a04e66fcf26 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:17.353601 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3e0ad847-ee7e-4325-abaf-0a04e66fcf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:17.396939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29db31e-62a2-4908-a1b1-aaca22214825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:17.397610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29db31e-62a2-4908-a1b1-aaca22214825 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:17.397882 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29db31e-62a2-4908-a1b1-aaca22214825 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:17.398147 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29db31e-62a2-4908-a1b1-aaca22214825 None admin] PATH_INFO: `/v3/users/405b6e24c00b4270baac3c351f0560c1` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:17.432840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29db31e-62a2-4908-a1b1-aaca22214825 None admin] RBAC: Authorizing `identity:get_user(user_id=405b6e24c00b4270baac3c351f0560c1)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:17.434343 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29db31e-62a2-4908-a1b1-aaca22214825 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:17.436430 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 121/244] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:17 2022] GET /identity/v3/users/405b6e24c00b4270baac3c351f0560c1 => generated 297 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:17.442162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29db31e-62a2-4908-a1b1-aaca22214825 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:17.442477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 124/245] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:17.601690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29db31e-62a2-4908-a1b1-aaca22214825 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:17.629712 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:17.652728 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba307e12-57a1-4f75-8481-9e3fda540106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:17.653394 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba307e12-57a1-4f75-8481-9e3fda540106 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:17.653663 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba307e12-57a1-4f75-8481-9e3fda540106 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:17.653926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba307e12-57a1-4f75-8481-9e3fda540106 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:17.730848 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba307e12-57a1-4f75-8481-9e3fda540106 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:17.732418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba307e12-57a1-4f75-8481-9e3fda540106 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:17.773613 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398cc54-cf67-4db1-8d85-290672ff51a2 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:17.773940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398cc54-cf67-4db1-8d85-290672ff51a2 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:17.774148 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398cc54-cf67-4db1-8d85-290672ff51a2 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:17.775548 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 125/246] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:54:17.780997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbeff2-23a3-4fc2-9178-7f3940aff552 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:17.781277 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbeff2-23a3-4fc2-9178-7f3940aff552 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:17.781541 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbbeff2-23a3-4fc2-9178-7f3940aff552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:17.811411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-afbbeff2-23a3-4fc2-9178-7f3940aff552 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:17.812680 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-afbbeff2-23a3-4fc2-9178-7f3940aff552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:17.856922 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 122/247] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:17.857710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba307e12-57a1-4f75-8481-9e3fda540106 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:17.877829 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:17.901743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 126/248] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:17.903048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f2696-688d-4ce0-af51-5167860ffd2d None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:17.903329 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f2696-688d-4ce0-af51-5167860ffd2d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:17.903785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f2696-688d-4ce0-af51-5167860ffd2d None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:17.905305 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 127/249] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:17.906029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f2696-688d-4ce0-af51-5167860ffd2d None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:17.937527 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:17.986662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba307e12-57a1-4f75-8481-9e3fda540106 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:18.026852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4149c5b4-5547-4b4b-a20b-e4d83b2b2a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:18.027775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149c5b4-5547-4b4b-a20b-e4d83b2b2a9e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:18.028048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149c5b4-5547-4b4b-a20b-e4d83b2b2a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:18.028324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4149c5b4-5547-4b4b-a20b-e4d83b2b2a9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:18.030164 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f2696-688d-4ce0-af51-5167860ffd2d None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:18.041086 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4149c5b4-5547-4b4b-a20b-e4d83b2b2a9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-4149c5b4-5547-4b4b-a20b-e4d83b2b2a9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:18.047939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:18.048859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:18.049689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:18.049689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:18.049689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:18.049689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:18.049689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:18.049689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:18.049689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:18.049689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:18.049689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:18.049689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:18.049689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:54:18.049689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:18.049689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 123/250] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:54:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:18.050924 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff83834-5053-4c56-ae16-035fba5a8f0d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:18.052805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff83834-5053-4c56-ae16-035fba5a8f0d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:18.052805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff83834-5053-4c56-ae16-035fba5a8f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:18.066208 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa1da5-6f48-4720-bd75-0f20ea614f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:18.066849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa1da5-6f48-4720-bd75-0f20ea614f5f None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:18.067229 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa1da5-6f48-4720-bd75-0f20ea614f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:18.067535 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa1da5-6f48-4720-bd75-0f20ea614f5f None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:18.068661 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa1da5-6f48-4720-bd75-0f20ea614f5f None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:18.070159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa1da5-6f48-4720-bd75-0f20ea614f5f None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:18.074836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 128/251] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:17 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:18.074836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa1da5-6f48-4720-bd75-0f20ea614f5f None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:18.079664 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-fff83834-5053-4c56-ae16-035fba5a8f0d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:18.081660 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-fff83834-5053-4c56-ae16-035fba5a8f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:18.164314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 124/252] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:18.165141 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff83834-5053-4c56-ae16-035fba5a8f0d None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:18.183882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:18.184786 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:18.195031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa1da5-6f48-4720-bd75-0f20ea614f5f None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:18.247653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2f4c4-7f6c-4b01-a92e-6df7bb1770c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:18.248432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2f4c4-7f6c-4b01-a92e-6df7bb1770c7 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:18.255477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2f4c4-7f6c-4b01-a92e-6df7bb1770c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:18.255477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a2f4c4-7f6c-4b01-a92e-6df7bb1770c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:18.255477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2f4c4-7f6c-4b01-a92e-6df7bb1770c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:18.255477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2f4c4-7f6c-4b01-a92e-6df7bb1770c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:18.255477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a2f4c4-7f6c-4b01-a92e-6df7bb1770c7 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:18.273277 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 129/253] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:18.278874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2f4c4-7f6c-4b01-a92e-6df7bb1770c7 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:18.345392 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff83834-5053-4c56-ae16-035fba5a8f0d None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:18.390305 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a2f4c4-7f6c-4b01-a92e-6df7bb1770c7 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:18.415861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0f722-c025-4a54-a407-34b5dbdec3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:18.416399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0f722-c025-4a54-a407-34b5dbdec3fb None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:18.416673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0f722-c025-4a54-a407-34b5dbdec3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:18.416948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0f722-c025-4a54-a407-34b5dbdec3fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:18.417688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0f722-c025-4a54-a407-34b5dbdec3fb None admin] RBAC: Adding query filter params (role.id=a9c50f8878284257bb064b71a6f7870b, scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd, user.id=405b6e24c00b4270baac3c351f0560c1) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:18.417993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0f722-c025-4a54-a407-34b5dbdec3fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:18.419468 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0f722-c025-4a54-a407-34b5dbdec3fb None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:18.426912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981c1c4-acd7-4a3b-b50b-6d13be45a271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:18.427582 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981c1c4-acd7-4a3b-b50b-6d13be45a271 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:18.427855 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981c1c4-acd7-4a3b-b50b-6d13be45a271 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:18.428122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981c1c4-acd7-4a3b-b50b-6d13be45a271 None admin] PATH_INFO: `/v3/users/8f442bafd9f1464f9d625ffec27c57e7` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:18.429260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981c1c4-acd7-4a3b-b50b-6d13be45a271 None admin] RBAC: Authorizing `identity:get_user(user_id=8f442bafd9f1464f9d625ffec27c57e7)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:18.430521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 125/254] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:18 2022] GET /identity/v3/role_assignments?user.id=405b6e24c00b4270baac3c351f0560c1&scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd&role.id=a9c50f8878284257bb064b71a6f7870b => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:18.431100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e981c1c4-acd7-4a3b-b50b-6d13be45a271 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:18.431345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0f722-c025-4a54-a407-34b5dbdec3fb None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:18.433138 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 130/255] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:18 2022] GET /identity/v3/users/8f442bafd9f1464f9d625ffec27c57e7 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:18.439243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981c1c4-acd7-4a3b-b50b-6d13be45a271 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:18.487906 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:18.595871 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:18.604336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981c1c4-acd7-4a3b-b50b-6d13be45a271 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:18.663858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0f722-c025-4a54-a407-34b5dbdec3fb None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:18.665715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e33e4e-6ac0-4ac9-8b77-946a78ebae07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:18.666401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e33e4e-6ac0-4ac9-8b77-946a78ebae07 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:18.666615 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e33e4e-6ac0-4ac9-8b77-946a78ebae07 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:18.666877 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e33e4e-6ac0-4ac9-8b77-946a78ebae07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:18.718163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb12b8-5427-469d-b953-a69912d2ef1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:18.718764 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb12b8-5427-469d-b953-a69912d2ef1e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:18.719040 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb12b8-5427-469d-b953-a69912d2ef1e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:18.719305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb12b8-5427-469d-b953-a69912d2ef1e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:18.744696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e33e4e-6ac0-4ac9-8b77-946a78ebae07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:18.746455 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e33e4e-6ac0-4ac9-8b77-946a78ebae07 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:18.748376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb12b8-5427-469d-b953-a69912d2ef1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-10eb12b8-5427-469d-b953-a69912d2ef1e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:18.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:18.754992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:18.760206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:18.760206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:18.760206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:18.760206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:18.760206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:18.760206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:18.760206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:18.760206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:18.760206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:18.760206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:18.760206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:18.760206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:18.760206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 126/256] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:54:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:18.760206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb12b8-5427-469d-b953-a69912d2ef1e None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:18.780194 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:18.859393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 131/257] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:18.861030 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e33e4e-6ac0-4ac9-8b77-946a78ebae07 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:18.970359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e33e4e-6ac0-4ac9-8b77-946a78ebae07 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:18.998406 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:19.009316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2b706-5ce7-49f1-b8d6-21a32fa214a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:19.010239 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2b706-5ce7-49f1-b8d6-21a32fa214a1 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:19.010748 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2b706-5ce7-49f1-b8d6-21a32fa214a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:19.011153 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2b706-5ce7-49f1-b8d6-21a32fa214a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:19.011989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2b706-5ce7-49f1-b8d6-21a32fa214a1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:19.012480 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2b706-5ce7-49f1-b8d6-21a32fa214a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:19.014219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b2b706-5ce7-49f1-b8d6-21a32fa214a1 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:19.023375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 132/258] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:19.024655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2b706-5ce7-49f1-b8d6-21a32fa214a1 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:19.025025 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb12b8-5427-469d-b953-a69912d2ef1e None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:19.069249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:19.088506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7ffec-8250-4b39-aee0-8768a4038072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:19.089134 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7ffec-8250-4b39-aee0-8768a4038072 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:19.089406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7ffec-8250-4b39-aee0-8768a4038072 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:19.089670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7ffec-8250-4b39-aee0-8768a4038072 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:19.109528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7ffec-8250-4b39-aee0-8768a4038072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-e1e7ffec-8250-4b39-aee0-8768a4038072 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:19.114485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:19.115583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:19.116665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:19.116665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:19.116665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:19.116665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:19.116665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:19.116665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:19.116665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:19.116665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:19.116665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:19.116665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:19.116665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:19.116665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:19.122822 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 127/259] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:19.123715 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7ffec-8250-4b39-aee0-8768a4038072 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:19.138891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b2b706-5ce7-49f1-b8d6-21a32fa214a1 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:19.176990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b0b69-8274-4e71-a0bf-35e812e6b4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:19.178035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b0b69-8274-4e71-a0bf-35e812e6b4da None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:19.178492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b0b69-8274-4e71-a0bf-35e812e6b4da None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:19.179212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b0b69-8274-4e71-a0bf-35e812e6b4da None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:19.182270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b0b69-8274-4e71-a0bf-35e812e6b4da None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:19.184804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b0b69-8274-4e71-a0bf-35e812e6b4da None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:19.186799 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 133/260] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:19 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:19.188048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b0b69-8274-4e71-a0bf-35e812e6b4da None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:19.259897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:19.310807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b0b69-8274-4e71-a0bf-35e812e6b4da None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:19.333360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7ffec-8250-4b39-aee0-8768a4038072 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:19.348657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad77934-f8df-4dc3-8170-f87946188d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:19.349632 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad77934-f8df-4dc3-8170-f87946188d42 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:19.350101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad77934-f8df-4dc3-8170-f87946188d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:19.350585 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad77934-f8df-4dc3-8170-f87946188d42 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:19.351500 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad77934-f8df-4dc3-8170-f87946188d42 None admin] RBAC: Adding query filter params (name=member) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:19.351989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad77934-f8df-4dc3-8170-f87946188d42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:19.354041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad77934-f8df-4dc3-8170-f87946188d42 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:19.363767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 134/261] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:19.365050 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad77934-f8df-4dc3-8170-f87946188d42 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:19.462028 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79876a-0ec2-41e9-94b5-506557029afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:19.463190 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79876a-0ec2-41e9-94b5-506557029afd None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:19.463373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79876a-0ec2-41e9-94b5-506557029afd None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:19.463636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79876a-0ec2-41e9-94b5-506557029afd None admin] PATH_INFO: `/v3/users/8f442bafd9f1464f9d625ffec27c57e7` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:19.464805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79876a-0ec2-41e9-94b5-506557029afd None admin] RBAC: Authorizing `identity:get_user(user_id=8f442bafd9f1464f9d625ffec27c57e7)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:19.466273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79876a-0ec2-41e9-94b5-506557029afd None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:19.472137 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad77934-f8df-4dc3-8170-f87946188d42 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:19.475907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 128/262] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:19 2022] GET /identity/v3/users/8f442bafd9f1464f9d625ffec27c57e7 => generated 283 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:19.477039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79876a-0ec2-41e9-94b5-506557029afd None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:19.511552 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:19.515680 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aedd52-fa6a-4cd9-86c0-1e26a1944d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:19.516308 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aedd52-fa6a-4cd9-86c0-1e26a1944d8a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:19.516591 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aedd52-fa6a-4cd9-86c0-1e26a1944d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:19.518216 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aedd52-fa6a-4cd9-86c0-1e26a1944d8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:19.518944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aedd52-fa6a-4cd9-86c0-1e26a1944d8a None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd, user.id=8f442bafd9f1464f9d625ffec27c57e7) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:19.519259 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aedd52-fa6a-4cd9-86c0-1e26a1944d8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:19.520756 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aedd52-fa6a-4cd9-86c0-1e26a1944d8a None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:19.533750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 135/263] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:19 2022] GET /identity/v3/role_assignments?user.id=8f442bafd9f1464f9d625ffec27c57e7&scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:19.535173 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aedd52-fa6a-4cd9-86c0-1e26a1944d8a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:19.645115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:19.724939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79876a-0ec2-41e9-94b5-506557029afd None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:19.794716 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9560da-76c4-4e47-a110-78b2047370fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:19.795375 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9560da-76c4-4e47-a110-78b2047370fd None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:19.795678 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9560da-76c4-4e47-a110-78b2047370fd None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:19.803236 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9560da-76c4-4e47-a110-78b2047370fd None admin] PATH_INFO: `/v3/users/80e27756f7664a16a1d86c3e886e93c9` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:19.804507 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9560da-76c4-4e47-a110-78b2047370fd None admin] RBAC: Authorizing `identity:get_user(user_id=80e27756f7664a16a1d86c3e886e93c9)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:19.806838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9560da-76c4-4e47-a110-78b2047370fd None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:19.807694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 129/264] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:19 2022] GET /identity/v3/users/80e27756f7664a16a1d86c3e886e93c9 => generated 297 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:19.813320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9560da-76c4-4e47-a110-78b2047370fd None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:19.814122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aedd52-fa6a-4cd9-86c0-1e26a1944d8a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:19.857369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34e8a0-0e13-4dbe-8a35-c019b16c43d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:19.858236 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34e8a0-0e13-4dbe-8a35-c019b16c43d5 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:19.858655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34e8a0-0e13-4dbe-8a35-c019b16c43d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:19.859088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34e8a0-0e13-4dbe-8a35-c019b16c43d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:19.930261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:19.964762 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34e8a0-0e13-4dbe-8a35-c019b16c43d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:19.966250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34e8a0-0e13-4dbe-8a35-c019b16c43d5 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:20.001372 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9560da-76c4-4e47-a110-78b2047370fd None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:20.047430 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:20.065910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a827a75-3e60-49ed-9c03-7254007e206d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:20.066918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a827a75-3e60-49ed-9c03-7254007e206d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:20.067714 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a827a75-3e60-49ed-9c03-7254007e206d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:20.068183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a827a75-3e60-49ed-9c03-7254007e206d None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:20.071318 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a827a75-3e60-49ed-9c03-7254007e206d None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:20.073801 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a827a75-3e60-49ed-9c03-7254007e206d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:20.076997 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 130/265] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:19 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:20.087271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a827a75-3e60-49ed-9c03-7254007e206d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:20.120658 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 136/266] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:20.126343 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34e8a0-0e13-4dbe-8a35-c019b16c43d5 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:20.249864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a827a75-3e60-49ed-9c03-7254007e206d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:20.283102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:20.308244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114dcf9-4457-44ad-a8c2-f0319a004953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:20.308689 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34e8a0-0e13-4dbe-8a35-c019b16c43d5 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:20.308862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114dcf9-4457-44ad-a8c2-f0319a004953 None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:20.309115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114dcf9-4457-44ad-a8c2-f0319a004953 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:20.309387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114dcf9-4457-44ad-a8c2-f0319a004953 None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd/users/80e27756f7664a16a1d86c3e886e93c9/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:20.320213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114dcf9-4457-44ad-a8c2-f0319a004953 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb5f153291f4fb28a4ccf24849d2ddd, user_id=80e27756f7664a16a1d86c3e886e93c9, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:20.321688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114dcf9-4457-44ad-a8c2-f0319a004953 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:20.350966 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb2fd8-4f60-44a7-8410-4d659e63bf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:20.351351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 131/267] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:54:20 2022] PUT /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd/users/80e27756f7664a16a1d86c3e886e93c9/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 15:54:20.352462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb2fd8-4f60-44a7-8410-4d659e63bf6d None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:20.352975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb2fd8-4f60-44a7-8410-4d659e63bf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:20.353440 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb2fd8-4f60-44a7-8410-4d659e63bf6d None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:20.355142 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb2fd8-4f60-44a7-8410-4d659e63bf6d None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:20.357122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb2fd8-4f60-44a7-8410-4d659e63bf6d None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:20.359899 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 137/268] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:20 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:20.367529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114dcf9-4457-44ad-a8c2-f0319a004953 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:20.462628 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:20.530581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114dcf9-4457-44ad-a8c2-f0319a004953 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:20.538723 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db3fd6-abc5-419d-bb1f-cbfd4fad908c None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:20.539063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db3fd6-abc5-419d-bb1f-cbfd4fad908c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:20.539272 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0db3fd6-abc5-419d-bb1f-cbfd4fad908c None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:20.541031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 138/269] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:20.552400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e3bb5-be81-4042-a106-80de8ad81518 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:20.552709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e3bb5-be81-4042-a106-80de8ad81518 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:20.552948 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e3bb5-be81-4042-a106-80de8ad81518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:20.571328 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a2dc0-5bc1-40d3-8533-3dd954bc9f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:20.571888 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a2dc0-5bc1-40d3-8533-3dd954bc9f56 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:20.572164 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a2dc0-5bc1-40d3-8533-3dd954bc9f56 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:20.572427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a2dc0-5bc1-40d3-8533-3dd954bc9f56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:20.573364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a2dc0-5bc1-40d3-8533-3dd954bc9f56 None admin] RBAC: Adding query filter params (role.id=bf98c97818504bc48e3bd1b2aef9d3c3, scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd, user.id=8f442bafd9f1464f9d625ffec27c57e7) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:20.573675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a2dc0-5bc1-40d3-8533-3dd954bc9f56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:20.576068 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a2dc0-5bc1-40d3-8533-3dd954bc9f56 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:20.583057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 132/270] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:20 2022] GET /identity/v3/role_assignments?user.id=8f442bafd9f1464f9d625ffec27c57e7&scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd&role.id=bf98c97818504bc48e3bd1b2aef9d3c3 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:20.591878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-e30e3bb5-be81-4042-a106-80de8ad81518 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:20.593104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e30e3bb5-be81-4042-a106-80de8ad81518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:20.613142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448460a-0f75-400a-b7c1-9670162ff5ec None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:20.613483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448460a-0f75-400a-b7c1-9670162ff5ec None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:20.613866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448460a-0f75-400a-b7c1-9670162ff5ec None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:20.615702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 133/271] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:54:20.621409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1194884-d32e-42f8-ad97-25aecddc04bb None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:20.621709 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1194884-d32e-42f8-ad97-25aecddc04bb None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:20.621960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1194884-d32e-42f8-ad97-25aecddc04bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:20.649180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f1194884-d32e-42f8-ad97-25aecddc04bb None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:20.650386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f1194884-d32e-42f8-ad97-25aecddc04bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:20.688535 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 139/272] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:20.740859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 134/273] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:21.017223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e3bb5-be81-4042-a106-80de8ad81518 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:21.053556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b910e6-a78a-41f4-9d72-1d24bfcfa9d2 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:21.053883 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b910e6-a78a-41f4-9d72-1d24bfcfa9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:21.054103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b910e6-a78a-41f4-9d72-1d24bfcfa9d2 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:21.055762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 135/274] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:21.060744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008e5a2-be07-4687-98d4-1f165e14a7d1 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:21.061021 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008e5a2-be07-4687-98d4-1f165e14a7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:21.061286 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008e5a2-be07-4687-98d4-1f165e14a7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:21.084110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-6008e5a2-be07-4687-98d4-1f165e14a7d1 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:21.085667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6008e5a2-be07-4687-98d4-1f165e14a7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:21.112945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:21.139251 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:21.140134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e3bb5-be81-4042-a106-80de8ad81518 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:21.165324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 136/275] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:21.168645 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008e5a2-be07-4687-98d4-1f165e14a7d1 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:21.183575 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c75f7-3e7e-4651-9e2e-5afbb5b53489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:21.184365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c75f7-3e7e-4651-9e2e-5afbb5b53489 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:21.184738 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c75f7-3e7e-4651-9e2e-5afbb5b53489 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:21.185096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c75f7-3e7e-4651-9e2e-5afbb5b53489 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:21.186386 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c75f7-3e7e-4651-9e2e-5afbb5b53489 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:21.188117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c75f7-3e7e-4651-9e2e-5afbb5b53489 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:21.190155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 140/276] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:21.195547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c75f7-3e7e-4651-9e2e-5afbb5b53489 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:21.301490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c75f7-3e7e-4651-9e2e-5afbb5b53489 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:21.306898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008e5a2-be07-4687-98d4-1f165e14a7d1 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:21.336643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8e086-dd3a-429e-9810-ca3af7be6fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:21.337405 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8e086-dd3a-429e-9810-ca3af7be6fde None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:21.338152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8e086-dd3a-429e-9810-ca3af7be6fde None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:21.338488 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8e086-dd3a-429e-9810-ca3af7be6fde None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:21.339231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8e086-dd3a-429e-9810-ca3af7be6fde None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:21.340784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8e086-dd3a-429e-9810-ca3af7be6fde None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:21.356143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480cf6a-d80e-4549-afe1-00fbb3dd06c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:21.357219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480cf6a-d80e-4549-afe1-00fbb3dd06c8 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:21.357860 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480cf6a-d80e-4549-afe1-00fbb3dd06c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:21.358453 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480cf6a-d80e-4549-afe1-00fbb3dd06c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:21.362489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:21.372817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 141/277] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:54:21 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:54:21.377452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480cf6a-d80e-4549-afe1-00fbb3dd06c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-7480cf6a-d80e-4549-afe1-00fbb3dd06c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:21.383436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:21.384540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:21.385369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:21.385369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:21.385369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:21.385369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:21.385369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:21.385369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:21.385369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:21.385369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:21.385369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:21.385369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:21.385369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:21.385369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:21.387751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 137/278] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:21.394209 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8e086-dd3a-429e-9810-ca3af7be6fde None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:21.467494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480cf6a-d80e-4549-afe1-00fbb3dd06c8 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:21.515762 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8e086-dd3a-429e-9810-ca3af7be6fde None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:21.524966 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:21.561479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb460ea-407f-4487-988e-e8eda1ff6a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:21.562012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb460ea-407f-4487-988e-e8eda1ff6a93 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:21.562292 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb460ea-407f-4487-988e-e8eda1ff6a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:21.562558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb460ea-407f-4487-988e-e8eda1ff6a93 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:21.563244 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb460ea-407f-4487-988e-e8eda1ff6a93 None admin] RBAC: Adding query filter params (name=member) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:21.563567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb460ea-407f-4487-988e-e8eda1ff6a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:21.565158 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb460ea-407f-4487-988e-e8eda1ff6a93 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:21.576993 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 142/279] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:21.582570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb460ea-407f-4487-988e-e8eda1ff6a93 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:21.618753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480cf6a-d80e-4549-afe1-00fbb3dd06c8 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:21.656582 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:21.664809 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798e4ac-a090-44e9-b26d-8878e12c350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:21.665461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798e4ac-a090-44e9-b26d-8878e12c350e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:21.665736 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798e4ac-a090-44e9-b26d-8878e12c350e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:21.666008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798e4ac-a090-44e9-b26d-8878e12c350e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:21.678903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798e4ac-a090-44e9-b26d-8878e12c350e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-7798e4ac-a090-44e9-b26d-8878e12c350e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:21.683879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:21.684916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:21.685887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:21.685887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:21.685887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:21.685887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:21.685887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:21.685887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:21.685887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:21.685887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:21.685887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:21.685887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:21.685887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:21.685887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:21.690053 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 138/280] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:54:21.690986 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798e4ac-a090-44e9-b26d-8878e12c350e None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:21.709277 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb460ea-407f-4487-988e-e8eda1ff6a93 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:21.754048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ced63-94d6-4b8d-80d1-8a6585a3f946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:21.755062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ced63-94d6-4b8d-80d1-8a6585a3f946 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:21.755691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ced63-94d6-4b8d-80d1-8a6585a3f946 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:21.755976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9ced63-94d6-4b8d-80d1-8a6585a3f946 None admin] PATH_INFO: `/v3/users/03df72eadf5140fe911855333a76c78a` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:21.776860 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ced63-94d6-4b8d-80d1-8a6585a3f946 None admin] RBAC: Authorizing `identity:get_user(user_id=03df72eadf5140fe911855333a76c78a)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:21.778354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9ced63-94d6-4b8d-80d1-8a6585a3f946 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:21.780282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 143/281] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:21 2022] GET /identity/v3/users/03df72eadf5140fe911855333a76c78a => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:21.786128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ced63-94d6-4b8d-80d1-8a6585a3f946 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:21.806434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:21.811000 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798e4ac-a090-44e9-b26d-8878e12c350e None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:21.851107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8847d03-ac4c-4426-be57-75c458fa6b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:21.852274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8847d03-ac4c-4426-be57-75c458fa6b89 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:21.852274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8847d03-ac4c-4426-be57-75c458fa6b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:21.852274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8847d03-ac4c-4426-be57-75c458fa6b89 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:21.852966 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8847d03-ac4c-4426-be57-75c458fa6b89 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:21.854348 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8847d03-ac4c-4426-be57-75c458fa6b89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:21.855524 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8847d03-ac4c-4426-be57-75c458fa6b89 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:21.864562 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 139/282] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:21.869985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8847d03-ac4c-4426-be57-75c458fa6b89 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:21.919899 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9ced63-94d6-4b8d-80d1-8a6585a3f946 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:21.948630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:21.972234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846fe65-8bff-49fc-85ed-06cabb8c611c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:21.973323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846fe65-8bff-49fc-85ed-06cabb8c611c None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:21.973763 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846fe65-8bff-49fc-85ed-06cabb8c611c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:21.974434 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f846fe65-8bff-49fc-85ed-06cabb8c611c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:21.976073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f846fe65-8bff-49fc-85ed-06cabb8c611c None admin] RBAC: Adding query filter params (role.id=abbd88ef58a544db8ee4261548778980, user.id=03df72eadf5140fe911855333a76c78a, scope.system=all) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:21.976660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f846fe65-8bff-49fc-85ed-06cabb8c611c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:21.978876 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f846fe65-8bff-49fc-85ed-06cabb8c611c None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:21.981215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8847d03-ac4c-4426-be57-75c458fa6b89 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:21.986549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 144/283] 10.222.0.27 () {56 vars in 1308 bytes} [Thu Sep 8 15:54:21 2022] GET /identity/v3/role_assignments?user.id=03df72eadf5140fe911855333a76c78a&scope.system=all&role.id=abbd88ef58a544db8ee4261548778980 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:22.021764 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03de0a-5382-4b49-a055-0e354b86b4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:22.022688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03de0a-5382-4b49-a055-0e354b86b4ef None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:22.023103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03de0a-5382-4b49-a055-0e354b86b4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:22.023531 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae03de0a-5382-4b49-a055-0e354b86b4ef None admin] PATH_INFO: `/v3/users/405b6e24c00b4270baac3c351f0560c1` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:22.024791 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03de0a-5382-4b49-a055-0e354b86b4ef None admin] RBAC: Authorizing `identity:get_user(user_id=405b6e24c00b4270baac3c351f0560c1)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:22.026402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae03de0a-5382-4b49-a055-0e354b86b4ef None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:22.028253 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 140/284] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:21 2022] GET /identity/v3/users/405b6e24c00b4270baac3c351f0560c1 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:22.037000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846fe65-8bff-49fc-85ed-06cabb8c611c None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:22.194422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f846fe65-8bff-49fc-85ed-06cabb8c611c None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:22.246226 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a97d29-20bb-4d72-aca9-06c46e7b3fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:22.246880 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a97d29-20bb-4d72-aca9-06c46e7b3fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:22.247154 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a97d29-20bb-4d72-aca9-06c46e7b3fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:22.247426 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a97d29-20bb-4d72-aca9-06c46e7b3fb0 None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:22.248625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a97d29-20bb-4d72-aca9-06c46e7b3fb0 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:22.250067 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a97d29-20bb-4d72-aca9-06c46e7b3fb0 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:22.252054 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 145/285] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:22 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:22.257845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03de0a-5382-4b49-a055-0e354b86b4ef None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:22.301667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:22.393570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae03de0a-5382-4b49-a055-0e354b86b4ef None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:22.439642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c575604-6563-452d-99bb-59d9f20e60bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:22.440761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c575604-6563-452d-99bb-59d9f20e60bf None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:22.441229 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c575604-6563-452d-99bb-59d9f20e60bf None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:22.441688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c575604-6563-452d-99bb-59d9f20e60bf None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd/users/405b6e24c00b4270baac3c351f0560c1/roles/a9c50f8878284257bb064b71a6f7870b` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:22.448772 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c575604-6563-452d-99bb-59d9f20e60bf None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb5f153291f4fb28a4ccf24849d2ddd, user_id=405b6e24c00b4270baac3c351f0560c1, role_id=a9c50f8878284257bb064b71a6f7870b)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:22.450484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c575604-6563-452d-99bb-59d9f20e60bf None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:22.478491 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 141/286] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:54:22 2022] PUT /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd/users/405b6e24c00b4270baac3c351f0560c1/roles/a9c50f8878284257bb064b71a6f7870b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:54:22.700592 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30956042-4cba-4677-b11f-69b219f3c111 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:22.701104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30956042-4cba-4677-b11f-69b219f3c111 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:22.701104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30956042-4cba-4677-b11f-69b219f3c111 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:22.702504 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 146/287] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:22.709118 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce571fb-a8ab-4dec-b54f-fd8c396e2878 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:22.709423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce571fb-a8ab-4dec-b54f-fd8c396e2878 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:22.709662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce571fb-a8ab-4dec-b54f-fd8c396e2878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:22.736599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-0ce571fb-a8ab-4dec-b54f-fd8c396e2878 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:22.737893 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0ce571fb-a8ab-4dec-b54f-fd8c396e2878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:22.809883 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 142/288] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:23.089164 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30956042-4cba-4677-b11f-69b219f3c111 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:23.154872 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:23.230237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30956042-4cba-4677-b11f-69b219f3c111 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:23.278815 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0245e2-a3a5-4d17-9e06-b9fa27642858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:23.279484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0245e2-a3a5-4d17-9e06-b9fa27642858 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:23.279755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0245e2-a3a5-4d17-9e06-b9fa27642858 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:23.280035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0245e2-a3a5-4d17-9e06-b9fa27642858 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:23.295581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0245e2-a3a5-4d17-9e06-b9fa27642858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-8e0245e2-a3a5-4d17-9e06-b9fa27642858 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:23.300685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:23.301558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:23.302432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:23.302432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:23.302432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:23.302432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:23.302432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:23.302432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:23.302432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:23.302432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:23.302432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:23.302432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:23.302432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:23.302432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:23.302432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 147/289] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:23.309327 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce571fb-a8ab-4dec-b54f-fd8c396e2878 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:23.339692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:23.447403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce571fb-a8ab-4dec-b54f-fd8c396e2878 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:23.489662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4894b-45a6-4a0d-a41d-96d16b61c957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:23.490288 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4894b-45a6-4a0d-a41d-96d16b61c957 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:23.490540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4894b-45a6-4a0d-a41d-96d16b61c957 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:23.490791 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d4894b-45a6-4a0d-a41d-96d16b61c957 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:23.491525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4894b-45a6-4a0d-a41d-96d16b61c957 None admin] RBAC: Adding query filter params (name=member) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:23.491827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4894b-45a6-4a0d-a41d-96d16b61c957 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:23.493209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d4894b-45a6-4a0d-a41d-96d16b61c957 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:23.507152 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 143/290] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:23.512605 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0245e2-a3a5-4d17-9e06-b9fa27642858 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:23.650204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0245e2-a3a5-4d17-9e06-b9fa27642858 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:23.666718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:23.690909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9ea3e-d14c-427e-a1f7-ca80add66991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:23.691638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9ea3e-d14c-427e-a1f7-ca80add66991 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:23.691920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9ea3e-d14c-427e-a1f7-ca80add66991 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:23.692193 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9ea3e-d14c-427e-a1f7-ca80add66991 None admin] PATH_INFO: `/v3/users/80e27756f7664a16a1d86c3e886e93c9` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:23.693340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9ea3e-d14c-427e-a1f7-ca80add66991 None admin] RBAC: Authorizing `identity:get_user(user_id=80e27756f7664a16a1d86c3e886e93c9)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:23.694806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d9ea3e-d14c-427e-a1f7-ca80add66991 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:23.696745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 148/291] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:23 2022] GET /identity/v3/users/80e27756f7664a16a1d86c3e886e93c9 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:23.702172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4894b-45a6-4a0d-a41d-96d16b61c957 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:23.823806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:23.846038 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d4894b-45a6-4a0d-a41d-96d16b61c957 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:23.881041 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21fdd2-abd5-4e15-bab3-36c119857607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:23.881653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21fdd2-abd5-4e15-bab3-36c119857607 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:23.881922 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21fdd2-abd5-4e15-bab3-36c119857607 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:23.882183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21fdd2-abd5-4e15-bab3-36c119857607 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:23.951936 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21fdd2-abd5-4e15-bab3-36c119857607 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:23.953332 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21fdd2-abd5-4e15-bab3-36c119857607 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:24.053625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:24.073560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 144/292] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:24.079282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9ea3e-d14c-427e-a1f7-ca80add66991 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:24.214192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d9ea3e-d14c-427e-a1f7-ca80add66991 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:24.250304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb19786-e846-4796-b2d0-0cdcf20c898c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:24.250942 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb19786-e846-4796-b2d0-0cdcf20c898c None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:24.251214 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb19786-e846-4796-b2d0-0cdcf20c898c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:24.251575 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb19786-e846-4796-b2d0-0cdcf20c898c None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:24.252710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb19786-e846-4796-b2d0-0cdcf20c898c None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:24.254182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb19786-e846-4796-b2d0-0cdcf20c898c None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:24.255861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 149/293] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:24 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:24.262071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21fdd2-abd5-4e15-bab3-36c119857607 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:24.320867 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a712ce5-aef9-4175-9815-734931a7bb43 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:24.321197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a712ce5-aef9-4175-9815-734931a7bb43 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:24.321420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a712ce5-aef9-4175-9815-734931a7bb43 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:24.322779 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 150/294] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:24.328726 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb24860-a9d2-4ee2-a297-c04b3d1abebe None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:24.329006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb24860-a9d2-4ee2-a297-c04b3d1abebe None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:24.329280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb24860-a9d2-4ee2-a297-c04b3d1abebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:24.359170 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-ccb24860-a9d2-4ee2-a297-c04b3d1abebe None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:24.360392 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ccb24860-a9d2-4ee2-a297-c04b3d1abebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:24.412207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21fdd2-abd5-4e15-bab3-36c119857607 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:24.443128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:24.474425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a159db-fb15-4800-87b6-95823139ecee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:24.475036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a159db-fb15-4800-87b6-95823139ecee None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:24.475310 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a159db-fb15-4800-87b6-95823139ecee None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:24.475599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a159db-fb15-4800-87b6-95823139ecee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:24.476600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a159db-fb15-4800-87b6-95823139ecee None admin] RBAC: Adding query filter params (role.id=abbd88ef58a544db8ee4261548778980, scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd, user.id=80e27756f7664a16a1d86c3e886e93c9) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:24.476900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a159db-fb15-4800-87b6-95823139ecee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:24.480237 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a159db-fb15-4800-87b6-95823139ecee None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:24.484584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 151/295] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:24.489619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 145/296] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:24 2022] GET /identity/v3/role_assignments?user.id=80e27756f7664a16a1d86c3e886e93c9&scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd&role.id=abbd88ef58a544db8ee4261548778980 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:24.708267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73812380-33d3-4f37-b3f0-874d20178cb3 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:24.708694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73812380-33d3-4f37-b3f0-874d20178cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:24.708901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73812380-33d3-4f37-b3f0-874d20178cb3 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:24.710570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 152/297] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:24.717017 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f610c-c977-4ce9-9e1c-71fc29a34214 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:24.717538 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f610c-c977-4ce9-9e1c-71fc29a34214 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:24.717538 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f610c-c977-4ce9-9e1c-71fc29a34214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:24.739219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-a57f610c-c977-4ce9-9e1c-71fc29a34214 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:24.740387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a57f610c-c977-4ce9-9e1c-71fc29a34214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:24.783516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:24.812538 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 146/298] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:24.849350 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73812380-33d3-4f37-b3f0-874d20178cb3 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:24.850644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4b19e-3573-4980-a821-221c787b2a33 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:24.850920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4b19e-3573-4980-a821-221c787b2a33 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:24.851194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4b19e-3573-4980-a821-221c787b2a33 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:24.852708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 147/299] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:24.858099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57b246-57c2-49d8-936e-abd5a58e86c1 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:24.858649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57b246-57c2-49d8-936e-abd5a58e86c1 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:24.859121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57b246-57c2-49d8-936e-abd5a58e86c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:24.878906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-be57b246-57c2-49d8-936e-abd5a58e86c1 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:24.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-be57b246-57c2-49d8-936e-abd5a58e86c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:24.959474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 148/300] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:24.966442 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73812380-33d3-4f37-b3f0-874d20178cb3 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:25.001748 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e2c88-e0b0-4bd6-ab93-3e86975c2e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:25.002308 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e2c88-e0b0-4bd6-ab93-3e86975c2e8b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:25.002632 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e2c88-e0b0-4bd6-ab93-3e86975c2e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:25.002903 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e2c88-e0b0-4bd6-ab93-3e86975c2e8b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:25.013925 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e2c88-e0b0-4bd6-ab93-3e86975c2e8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-c93e2c88-e0b0-4bd6-ab93-3e86975c2e8b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:25.019084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:25.020204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:25.021199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:25.021199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:25.021199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:25.021199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:25.021199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:25.021199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:25.021199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:25.021199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:25.021199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:25.021199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:25.021199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:25.021199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:25.023161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 153/301] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:25.029195 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57b246-57c2-49d8-936e-abd5a58e86c1 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:25.101647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e2c88-e0b0-4bd6-ab93-3e86975c2e8b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:25.149148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57b246-57c2-49d8-936e-abd5a58e86c1 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:25.151624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:25.154044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:25.213832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010409e-6b09-429b-8cb6-329f9d5e97b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:25.214995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010409e-6b09-429b-8cb6-329f9d5e97b6 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:25.215414 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010409e-6b09-429b-8cb6-329f9d5e97b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:25.215853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010409e-6b09-429b-8cb6-329f9d5e97b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:25.217553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010409e-6b09-429b-8cb6-329f9d5e97b6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:25.218206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010409e-6b09-429b-8cb6-329f9d5e97b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:25.221791 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010409e-6b09-429b-8cb6-329f9d5e97b6 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:25.238636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 149/302] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:25.244761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010409e-6b09-429b-8cb6-329f9d5e97b6 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:25.257171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e2c88-e0b0-4bd6-ab93-3e86975c2e8b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:25.291542 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9939e7-50b1-4875-b951-d9d2c6c8fa0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:25.292182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9939e7-50b1-4875-b951-d9d2c6c8fa0b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:25.292454 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9939e7-50b1-4875-b951-d9d2c6c8fa0b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:25.292719 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9939e7-50b1-4875-b951-d9d2c6c8fa0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:25.293817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9939e7-50b1-4875-b951-d9d2c6c8fa0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:25.295249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9939e7-50b1-4875-b951-d9d2c6c8fa0b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:25.296852 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 154/303] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:25.296852 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9939e7-50b1-4875-b951-d9d2c6c8fa0b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:25.355704 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010409e-6b09-429b-8cb6-329f9d5e97b6 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:25.388367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a4686-82c1-4a1d-b0bc-7843a217bc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:25.389009 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4686-82c1-4a1d-b0bc-7843a217bc5a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:25.389301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4686-82c1-4a1d-b0bc-7843a217bc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:25.389566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a4686-82c1-4a1d-b0bc-7843a217bc5a None admin] PATH_INFO: `/v3/users/03a5d01f5f104dbcaa197ee7900b81d0` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:25.395426 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9939e7-50b1-4875-b951-d9d2c6c8fa0b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:25.402461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a4686-82c1-4a1d-b0bc-7843a217bc5a None admin] RBAC: Authorizing `identity:get_user(user_id=03a5d01f5f104dbcaa197ee7900b81d0)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:25.402608 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:25.404007 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a4686-82c1-4a1d-b0bc-7843a217bc5a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:25.405884 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 150/304] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:25 2022] GET /identity/v3/users/03a5d01f5f104dbcaa197ee7900b81d0 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:25.407006 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaae6e0-50cb-4187-8021-aa8c67852883 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:25.407282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaae6e0-50cb-4187-8021-aa8c67852883 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:25.407584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaae6e0-50cb-4187-8021-aa8c67852883 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:25.408767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 151/305] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:25.409535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaae6e0-50cb-4187-8021-aa8c67852883 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:25.433852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:25.434677 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24163257-e5c4-4fdd-8e03-51d038ed632b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:25.435330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24163257-e5c4-4fdd-8e03-51d038ed632b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:25.435617 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24163257-e5c4-4fdd-8e03-51d038ed632b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:25.435911 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24163257-e5c4-4fdd-8e03-51d038ed632b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:25.446082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24163257-e5c4-4fdd-8e03-51d038ed632b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-24163257-e5c4-4fdd-8e03-51d038ed632b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:25.450878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:25.451789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:25.452566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:25.452566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:25.452566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:25.452566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:25.452566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:25.452566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:25.452566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:25.452566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:25.452566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:25.452566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:25.452566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:25.452566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:25.452566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 155/306] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:25.453229 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24163257-e5c4-4fdd-8e03-51d038ed632b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:25.508615 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaae6e0-50cb-4187-8021-aa8c67852883 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:25.542671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df507851-e4ab-4641-8c77-8c31b7d56aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:25.543296 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df507851-e4ab-4641-8c77-8c31b7d56aa3 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:25.543578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df507851-e4ab-4641-8c77-8c31b7d56aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:25.543842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df507851-e4ab-4641-8c77-8c31b7d56aa3 None admin] PATH_INFO: `/v3/groups` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:25.544479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df507851-e4ab-4641-8c77-8c31b7d56aa3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:25.545918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df507851-e4ab-4641-8c77-8c31b7d56aa3 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:25.553633 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24163257-e5c4-4fdd-8e03-51d038ed632b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:25.562316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 152/307] 10.222.0.27 () {60 vars in 1134 bytes} [Thu Sep 8 15:54:25 2022] POST /identity/v3/groups => generated 229 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:54:25.563610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a95210-53de-4c5c-86e2-45aef149310f None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:25.564170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a95210-53de-4c5c-86e2-45aef149310f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:25.564170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a95210-53de-4c5c-86e2-45aef149310f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:25.592190 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-b1a95210-53de-4c5c-86e2-45aef149310f None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:25.593553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b1a95210-53de-4c5c-86e2-45aef149310f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:25.593750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82c3b2-681b-4293-8029-e5a460459c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:25.594384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82c3b2-681b-4293-8029-e5a460459c36 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:25.594654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82c3b2-681b-4293-8029-e5a460459c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:25.594916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82c3b2-681b-4293-8029-e5a460459c36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:25.595705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82c3b2-681b-4293-8029-e5a460459c36 None admin] RBAC: Adding query filter params (role.id=a9c50f8878284257bb064b71a6f7870b, user.id=03a5d01f5f104dbcaa197ee7900b81d0, scope.system=all) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:25.596000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82c3b2-681b-4293-8029-e5a460459c36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:25.597522 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82c3b2-681b-4293-8029-e5a460459c36 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:25.615480 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 156/308] 10.222.0.27 () {56 vars in 1308 bytes} [Thu Sep 8 15:54:25 2022] GET /identity/v3/role_assignments?user.id=03a5d01f5f104dbcaa197ee7900b81d0&scope.system=all&role.id=a9c50f8878284257bb064b71a6f7870b => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:25.615480 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82c3b2-681b-4293-8029-e5a460459c36 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:25.658962 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:25.697559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 153/309] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:25.758427 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82c3b2-681b-4293-8029-e5a460459c36 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:25.798353 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455948e8-d8bd-455d-9b31-03d5ed069255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:25.798970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455948e8-d8bd-455d-9b31-03d5ed069255 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:25.799238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455948e8-d8bd-455d-9b31-03d5ed069255 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:25.799536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455948e8-d8bd-455d-9b31-03d5ed069255 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:25.800356 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455948e8-d8bd-455d-9b31-03d5ed069255 None admin] RBAC: Adding query filter params (name=member) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:25.800654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455948e8-d8bd-455d-9b31-03d5ed069255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:25.802010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455948e8-d8bd-455d-9b31-03d5ed069255 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:25.812576 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 157/310] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:25.818250 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a95210-53de-4c5c-86e2-45aef149310f None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:25.835113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:25.925729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a95210-53de-4c5c-86e2-45aef149310f None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:25.961569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423bae2-034d-4f11-9bd6-7fc94eae4695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:25.962035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455948e8-d8bd-455d-9b31-03d5ed069255 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:25.963088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423bae2-034d-4f11-9bd6-7fc94eae4695 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:25.963559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423bae2-034d-4f11-9bd6-7fc94eae4695 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:25.964010 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423bae2-034d-4f11-9bd6-7fc94eae4695 None admin] PATH_INFO: `/v3/users/03df72eadf5140fe911855333a76c78a` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:25.965321 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423bae2-034d-4f11-9bd6-7fc94eae4695 None admin] RBAC: Authorizing `identity:get_user(user_id=03df72eadf5140fe911855333a76c78a)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:25.966988 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2423bae2-034d-4f11-9bd6-7fc94eae4695 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:25.973575 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 154/311] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:25 2022] GET /identity/v3/users/03df72eadf5140fe911855333a76c78a => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:25.976185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423bae2-034d-4f11-9bd6-7fc94eae4695 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:26.079234 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:26.080213 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:26.091122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455948e8-d8bd-455d-9b31-03d5ed069255 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:26.105990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423bae2-034d-4f11-9bd6-7fc94eae4695 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:26.134682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df711b31-340e-4163-80b0-584d9f5bb064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:26.135585 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df711b31-340e-4163-80b0-584d9f5bb064 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:26.135962 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df711b31-340e-4163-80b0-584d9f5bb064 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:26.136571 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df711b31-340e-4163-80b0-584d9f5bb064 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:26.150959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df711b31-340e-4163-80b0-584d9f5bb064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:26.151693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ab009-5145-4f84-96d8-29da52ac342d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:26.152495 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ab009-5145-4f84-96d8-29da52ac342d None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:26.152856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ab009-5145-4f84-96d8-29da52ac342d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:26.153216 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ab009-5145-4f84-96d8-29da52ac342d None admin] PATH_INFO: `/v3/system/users/03df72eadf5140fe911855333a76c78a/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:26.154721 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ab009-5145-4f84-96d8-29da52ac342d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=03df72eadf5140fe911855333a76c78a, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:26.156231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ab009-5145-4f84-96d8-29da52ac342d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-df711b31-340e-4163-80b0-584d9f5bb064 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:26.161013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:26.162970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:26.163895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:26.163895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:26.163895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:26.163895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:26.163895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:26.163895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:26.163895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:26.163895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:26.163895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:26.163895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:26.163895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:26.163895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:26.173865 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 158/312] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:54:26.174025 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 155/313] 10.222.0.27 () {58 vars in 1334 bytes} [Thu Sep 8 15:54:25 2022] PUT /identity/v3/system/users/03df72eadf5140fe911855333a76c78a/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:54:26.176438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df711b31-340e-4163-80b0-584d9f5bb064 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:26.292974 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df711b31-340e-4163-80b0-584d9f5bb064 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:26.330833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447f2c1-b44c-45c9-999f-ed902e263669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:26.331563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447f2c1-b44c-45c9-999f-ed902e263669 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:26.331843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447f2c1-b44c-45c9-999f-ed902e263669 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:26.332120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447f2c1-b44c-45c9-999f-ed902e263669 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:26.332815 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447f2c1-b44c-45c9-999f-ed902e263669 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:26.333113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447f2c1-b44c-45c9-999f-ed902e263669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:26.334517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447f2c1-b44c-45c9-999f-ed902e263669 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:26.344836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 159/314] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:26.350820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ab009-5145-4f84-96d8-29da52ac342d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:26.454262 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ab009-5145-4f84-96d8-29da52ac342d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:26.488814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f2c05-7fb5-4e11-9d15-8be6ec91d683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:26.489479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f2c05-7fb5-4e11-9d15-8be6ec91d683 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:26.489751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f2c05-7fb5-4e11-9d15-8be6ec91d683 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:26.490017 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f2c05-7fb5-4e11-9d15-8be6ec91d683 None admin] PATH_INFO: `/v3/users/405b6e24c00b4270baac3c351f0560c1` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:26.491266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f2c05-7fb5-4e11-9d15-8be6ec91d683 None admin] RBAC: Authorizing `identity:get_user(user_id=405b6e24c00b4270baac3c351f0560c1)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:26.492817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f2c05-7fb5-4e11-9d15-8be6ec91d683 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:26.498196 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 156/315] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:26 2022] GET /identity/v3/users/405b6e24c00b4270baac3c351f0560c1 => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:26.501146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447f2c1-b44c-45c9-999f-ed902e263669 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:26.519016 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:26.618072 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447f2c1-b44c-45c9-999f-ed902e263669 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:26.655130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8d6a0-f287-41a6-991e-b81e49b4c5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:26.655772 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8d6a0-f287-41a6-991e-b81e49b4c5ed None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:26.656052 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8d6a0-f287-41a6-991e-b81e49b4c5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:26.656318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8d6a0-f287-41a6-991e-b81e49b4c5ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:26.724909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8d6a0-f287-41a6-991e-b81e49b4c5ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:26.726376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8d6a0-f287-41a6-991e-b81e49b4c5ed None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:26.741269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:26.825217 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 160/316] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:26.830921 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f2c05-7fb5-4e11-9d15-8be6ec91d683 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:26.848730 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:26.936502 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f2c05-7fb5-4e11-9d15-8be6ec91d683 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:26.970620 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a79da6d-73ef-4e09-8418-07d9e322f855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:26.971260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79da6d-73ef-4e09-8418-07d9e322f855 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:26.971560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79da6d-73ef-4e09-8418-07d9e322f855 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:26.971851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a79da6d-73ef-4e09-8418-07d9e322f855 None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:26.973063 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79da6d-73ef-4e09-8418-07d9e322f855 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:26.974514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a79da6d-73ef-4e09-8418-07d9e322f855 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:26.976650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 157/317] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:26 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:26.982270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8d6a0-f287-41a6-991e-b81e49b4c5ed None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:27.083675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8d6a0-f287-41a6-991e-b81e49b4c5ed None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:27.098220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:27.118551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6e7f6-c673-4414-a43a-156bc51dca68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:27.119162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6e7f6-c673-4414-a43a-156bc51dca68 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:27.119438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6e7f6-c673-4414-a43a-156bc51dca68 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:27.119734 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6e7f6-c673-4414-a43a-156bc51dca68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:27.120462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6e7f6-c673-4414-a43a-156bc51dca68 None admin] RBAC: Adding query filter params (role.id=a9c50f8878284257bb064b71a6f7870b, scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd, user.id=405b6e24c00b4270baac3c351f0560c1) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:27.120765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6e7f6-c673-4414-a43a-156bc51dca68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:27.122211 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6e7f6-c673-4414-a43a-156bc51dca68 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:27.128395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 161/318] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:26 2022] GET /identity/v3/role_assignments?user.id=405b6e24c00b4270baac3c351f0560c1&scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd&role.id=a9c50f8878284257bb064b71a6f7870b => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:27.777771 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644c5c8-b8a1-4384-bf6d-cd5854b6bb5e None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:27.778044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644c5c8-b8a1-4384-bf6d-cd5854b6bb5e None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:27.778324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644c5c8-b8a1-4384-bf6d-cd5854b6bb5e None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:27.779884 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 158/319] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:27.785033 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb54ef-bd56-4778-a97c-11c5ff0e47ca None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:27.785490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb54ef-bd56-4778-a97c-11c5ff0e47ca None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:27.785561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb54ef-bd56-4778-a97c-11c5ff0e47ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:27.804101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-21fb54ef-bd56-4778-a97c-11c5ff0e47ca None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:27.805280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-21fb54ef-bd56-4778-a97c-11c5ff0e47ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:27.848828 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca15f3b-3b89-446b-8b58-9ff29dc308ba None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:27.849347 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca15f3b-3b89-446b-8b58-9ff29dc308ba None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:27.849506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca15f3b-3b89-446b-8b58-9ff29dc308ba None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:27.850864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 159/320] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:27.856032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a02c46-a77f-416c-92e8-a421690b979c None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:27.856369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a02c46-a77f-416c-92e8-a421690b979c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:27.856742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a02c46-a77f-416c-92e8-a421690b979c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:27.874695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-31a02c46-a77f-416c-92e8-a421690b979c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:27.875914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-31a02c46-a77f-416c-92e8-a421690b979c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:27.876631 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 162/321] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:27.941982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 160/322] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:28.140388 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb54ef-bd56-4778-a97c-11c5ff0e47ca None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:28.201826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a02c46-a77f-416c-92e8-a421690b979c None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:28.216748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:28.247163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb54ef-bd56-4778-a97c-11c5ff0e47ca None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:28.258950 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:28.286237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1840e-8b38-46bf-993d-4d7f098d97f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:28.286877 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1840e-8b38-46bf-993d-4d7f098d97f8 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:28.287152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1840e-8b38-46bf-993d-4d7f098d97f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:28.287419 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1840e-8b38-46bf-993d-4d7f098d97f8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:28.298578 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1840e-8b38-46bf-993d-4d7f098d97f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-aff1840e-8b38-46bf-993d-4d7f098d97f8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:28.303276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:28.304162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:28.305581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:28.305581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:28.305581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:28.305581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:28.305581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:28.305581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:28.305581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:28.305581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:28.305581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:28.305581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:28.305581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:28.305581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:28.305581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 163/323] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:28.310175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1840e-8b38-46bf-993d-4d7f098d97f8 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:28.310295 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a02c46-a77f-416c-92e8-a421690b979c None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:28.349341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcbd88-ce0b-4618-8f78-64555910e301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:28.350081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcbd88-ce0b-4618-8f78-64555910e301 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:28.350363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcbd88-ce0b-4618-8f78-64555910e301 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:28.350683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcbd88-ce0b-4618-8f78-64555910e301 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:28.351954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcbd88-ce0b-4618-8f78-64555910e301 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:28.353445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcbd88-ce0b-4618-8f78-64555910e301 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:28.355123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 161/324] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:28.360960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcbd88-ce0b-4618-8f78-64555910e301 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:28.423511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1840e-8b38-46bf-993d-4d7f098d97f8 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:28.446370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:28.459895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0fd98-a90d-45db-99b5-3677a5c49952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:28.460516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0fd98-a90d-45db-99b5-3677a5c49952 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:28.460794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0fd98-a90d-45db-99b5-3677a5c49952 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:28.461066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0fd98-a90d-45db-99b5-3677a5c49952 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:28.461739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0fd98-a90d-45db-99b5-3677a5c49952 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:28.462049 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0fd98-a90d-45db-99b5-3677a5c49952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:28.463836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0fd98-a90d-45db-99b5-3677a5c49952 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:28.467117 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcbd88-ce0b-4618-8f78-64555910e301 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:28.471966 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 164/325] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:28.473160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd33db-0c0a-48fd-b1ca-493858498352 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:28.473444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd33db-0c0a-48fd-b1ca-493858498352 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:28.473721 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddd33db-0c0a-48fd-b1ca-493858498352 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:28.474934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 165/326] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:28.478080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd33db-0c0a-48fd-b1ca-493858498352 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:28.503180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cbff6-0d77-4c13-ae29-adaab0a23376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:28.503898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cbff6-0d77-4c13-ae29-adaab0a23376 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:28.504176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cbff6-0d77-4c13-ae29-adaab0a23376 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:28.504447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cbff6-0d77-4c13-ae29-adaab0a23376 None admin] PATH_INFO: `/v3/groups` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:28.505083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cbff6-0d77-4c13-ae29-adaab0a23376 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:28.506523 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cbff6-0d77-4c13-ae29-adaab0a23376 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:28.508782 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:28.521500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 162/327] 10.222.0.27 () {60 vars in 1134 bytes} [Thu Sep 8 15:54:28 2022] POST /identity/v3/groups => generated 226 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 15:54:28.522698 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98ad8d-dcd6-4db7-b66b-f7a2d43a1afd None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:28.522978 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98ad8d-dcd6-4db7-b66b-f7a2d43a1afd None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:28.523245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98ad8d-dcd6-4db7-b66b-f7a2d43a1afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:28.542506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-1c98ad8d-dcd6-4db7-b66b-f7a2d43a1afd None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:28.543765 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-1c98ad8d-dcd6-4db7-b66b-f7a2d43a1afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:28.595031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddd33db-0c0a-48fd-b1ca-493858498352 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:28.623695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 163/328] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:28.639848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0da42-532c-4f56-a28f-3eb93f5fef7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:28.640510 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0da42-532c-4f56-a28f-3eb93f5fef7f None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:28.640782 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0da42-532c-4f56-a28f-3eb93f5fef7f None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:28.641050 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0da42-532c-4f56-a28f-3eb93f5fef7f None admin] PATH_INFO: `/v3/users/03a5d01f5f104dbcaa197ee7900b81d0` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:28.642431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0da42-532c-4f56-a28f-3eb93f5fef7f None admin] RBAC: Authorizing `identity:get_user(user_id=03a5d01f5f104dbcaa197ee7900b81d0)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:28.644020 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca0da42-532c-4f56-a28f-3eb93f5fef7f None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:28.650313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 166/329] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:28 2022] GET /identity/v3/users/03a5d01f5f104dbcaa197ee7900b81d0 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:28.652119 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98ad8d-dcd6-4db7-b66b-f7a2d43a1afd None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:28.733566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:28.767851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98ad8d-dcd6-4db7-b66b-f7a2d43a1afd None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:28.802859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7616b9f-75cc-4ead-a5ee-96101900e1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:28.803552 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7616b9f-75cc-4ead-a5ee-96101900e1d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:28.803877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7616b9f-75cc-4ead-a5ee-96101900e1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:28.804145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7616b9f-75cc-4ead-a5ee-96101900e1d7 None admin] PATH_INFO: `/v3/system/users/03a5d01f5f104dbcaa197ee7900b81d0/roles/a9c50f8878284257bb064b71a6f7870b` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:28.805555 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7616b9f-75cc-4ead-a5ee-96101900e1d7 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=03a5d01f5f104dbcaa197ee7900b81d0, role_id=a9c50f8878284257bb064b71a6f7870b)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:28.807035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7616b9f-75cc-4ead-a5ee-96101900e1d7 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:28.818203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 164/330] 10.222.0.27 () {58 vars in 1334 bytes} [Thu Sep 8 15:54:28 2022] PUT /identity/v3/system/users/03a5d01f5f104dbcaa197ee7900b81d0/roles/a9c50f8878284257bb064b71a6f7870b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:54:28.889872 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0da42-532c-4f56-a28f-3eb93f5fef7f None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:29.010929 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:29.016223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca0da42-532c-4f56-a28f-3eb93f5fef7f None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:29.054871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d9c32-89b7-42c5-bd7c-57287129d6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:29.055713 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d9c32-89b7-42c5-bd7c-57287129d6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:29.056143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d9c32-89b7-42c5-bd7c-57287129d6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:29.056520 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d9c32-89b7-42c5-bd7c-57287129d6f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:29.067821 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d9c32-89b7-42c5-bd7c-57287129d6f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-342d9c32-89b7-42c5-bd7c-57287129d6f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:29.073641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:29.074722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:29.075788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:29.075788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:29.075788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:29.075788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:29.075788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:29.075788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:29.075788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:29.075788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:29.075788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:29.075788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:29.075788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:29.075788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:29.077563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 167/331] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:54:29.083289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7616b9f-75cc-4ead-a5ee-96101900e1d7 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:29.200656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7616b9f-75cc-4ead-a5ee-96101900e1d7 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:29.236317 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0c5ef-df8e-42fa-b2fb-5334d0b6de51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:29.237085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0c5ef-df8e-42fa-b2fb-5334d0b6de51 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:29.237491 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0c5ef-df8e-42fa-b2fb-5334d0b6de51 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:29.237855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e0c5ef-df8e-42fa-b2fb-5334d0b6de51 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:29.238623 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0c5ef-df8e-42fa-b2fb-5334d0b6de51 None admin] RBAC: Adding query filter params (name=member) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:29.239018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0c5ef-df8e-42fa-b2fb-5334d0b6de51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:29.240938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e0c5ef-df8e-42fa-b2fb-5334d0b6de51 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:29.243062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:29.250826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 165/332] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:29.256487 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d9c32-89b7-42c5-bd7c-57287129d6f4 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:29.388090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d9c32-89b7-42c5-bd7c-57287129d6f4 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:29.429066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273829a-10ed-4d5d-92a9-e5a6441bc7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:29.429725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273829a-10ed-4d5d-92a9-e5a6441bc7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:29.429997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273829a-10ed-4d5d-92a9-e5a6441bc7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:29.430352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273829a-10ed-4d5d-92a9-e5a6441bc7e3 None admin] PATH_INFO: `/v3/users/03df72eadf5140fe911855333a76c78a` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:29.431730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273829a-10ed-4d5d-92a9-e5a6441bc7e3 None admin] RBAC: Authorizing `identity:get_user(user_id=03df72eadf5140fe911855333a76c78a)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:29.433187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273829a-10ed-4d5d-92a9-e5a6441bc7e3 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:29.435245 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 168/333] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:29 2022] GET /identity/v3/users/03df72eadf5140fe911855333a76c78a => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:29.440835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0c5ef-df8e-42fa-b2fb-5334d0b6de51 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:29.556182 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e0c5ef-df8e-42fa-b2fb-5334d0b6de51 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:29.597753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d31a7-9caa-4029-9e83-5d3ba5523bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:29.598581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d31a7-9caa-4029-9e83-5d3ba5523bff None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:29.598859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d31a7-9caa-4029-9e83-5d3ba5523bff None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:29.599133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d31a7-9caa-4029-9e83-5d3ba5523bff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:29.599918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d31a7-9caa-4029-9e83-5d3ba5523bff None admin] RBAC: Adding query filter params (role.id=abbd88ef58a544db8ee4261548778980, user.id=03df72eadf5140fe911855333a76c78a, scope.system=all) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:29.600228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d31a7-9caa-4029-9e83-5d3ba5523bff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:29.601726 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3d31a7-9caa-4029-9e83-5d3ba5523bff None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:29.616341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 166/334] 10.222.0.27 () {56 vars in 1308 bytes} [Thu Sep 8 15:54:29 2022] GET /identity/v3/role_assignments?user.id=03df72eadf5140fe911855333a76c78a&scope.system=all&role.id=abbd88ef58a544db8ee4261548778980 => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:31.015985 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ccc91-309f-4999-9756-b23a739829d1 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:31.016511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ccc91-309f-4999-9756-b23a739829d1 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:31.016511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ccc91-309f-4999-9756-b23a739829d1 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:31.018205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 169/335] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:31.023837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b727e7-b369-4bd5-816b-9cfac1ba06f0 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:31.024339 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b727e7-b369-4bd5-816b-9cfac1ba06f0 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:31.024412 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b727e7-b369-4bd5-816b-9cfac1ba06f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:31.052326 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-04b727e7-b369-4bd5-816b-9cfac1ba06f0 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:31.053535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-04b727e7-b369-4bd5-816b-9cfac1ba06f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:31.127102 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:31.144340 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 167/336] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:31.359930 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1258e-cd59-49ca-9f7b-85d5cfb9b850 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:31.360319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1258e-cd59-49ca-9f7b-85d5cfb9b850 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:31.360513 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f1258e-cd59-49ca-9f7b-85d5cfb9b850 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:31.361863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 170/337] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:31.367927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf9e76-b595-4480-8184-34f8fddb53c9 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:31.368214 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf9e76-b595-4480-8184-34f8fddb53c9 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:31.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebf9e76-b595-4480-8184-34f8fddb53c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:31.393520 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-cebf9e76-b595-4480-8184-34f8fddb53c9 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:31.394772 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-cebf9e76-b595-4480-8184-34f8fddb53c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:31.481489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 168/338] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:31.703801 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1258e-cd59-49ca-9f7b-85d5cfb9b850 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:31.737677 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:31.827846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f1258e-cd59-49ca-9f7b-85d5cfb9b850 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:31.878340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804cbdd-5370-4d40-8547-66af4c1df135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:31.878987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804cbdd-5370-4d40-8547-66af4c1df135 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:31.879266 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804cbdd-5370-4d40-8547-66af4c1df135 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:31.879567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804cbdd-5370-4d40-8547-66af4c1df135 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:31.893097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804cbdd-5370-4d40-8547-66af4c1df135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-4804cbdd-5370-4d40-8547-66af4c1df135 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:31.898096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:31.899000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:31.899840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:31.899840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:31.899840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:31.899840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:31.899840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:31.899840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:31.899840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:31.899840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:31.899840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:31.899840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:31.899840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:31.899840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:31.899840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 171/339] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:31.905161 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf9e76-b595-4480-8184-34f8fddb53c9 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:32.023530 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5203927-007a-4d9b-8774-13341b5ddc4d None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:32.024040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5203927-007a-4d9b-8774-13341b5ddc4d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:32.024134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5203927-007a-4d9b-8774-13341b5ddc4d None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:32.025441 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 172/340] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:32.032035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b29a9-cd3d-4288-9a58-3a00eb1e686a None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:32.032371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b29a9-cd3d-4288-9a58-3a00eb1e686a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:32.032589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1b29a9-cd3d-4288-9a58-3a00eb1e686a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:32.046374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebf9e76-b595-4480-8184-34f8fddb53c9 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:32.072040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-1b1b29a9-cd3d-4288-9a58-3a00eb1e686a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:32.073256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1b1b29a9-cd3d-4288-9a58-3a00eb1e686a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:32.081599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:32.122774 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e73c0d-6c44-4747-b7a7-73f34b465149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:32.123402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e73c0d-6c44-4747-b7a7-73f34b465149 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:32.123700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e73c0d-6c44-4747-b7a7-73f34b465149 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:32.123966 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e73c0d-6c44-4747-b7a7-73f34b465149 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:32.124646 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e73c0d-6c44-4747-b7a7-73f34b465149 None admin] RBAC: Adding query filter params (name=member) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:32.124946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e73c0d-6c44-4747-b7a7-73f34b465149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:32.126331 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e73c0d-6c44-4747-b7a7-73f34b465149 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:32.140930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 169/341] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:32.146387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e73c0d-6c44-4747-b7a7-73f34b465149 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:32.176835 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 173/342] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:32.178563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ff359-e147-4354-b0d7-ba22b7f0b532 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:32.179051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ff359-e147-4354-b0d7-ba22b7f0b532 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:32.179553 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ff359-e147-4354-b0d7-ba22b7f0b532 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:32.181280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 174/343] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:32.182177 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ff359-e147-4354-b0d7-ba22b7f0b532 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:32.202549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:32.328920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e73c0d-6c44-4747-b7a7-73f34b465149 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:32.357539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ff359-e147-4354-b0d7-ba22b7f0b532 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:32.378586 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fcd85-d4ea-4bbe-9441-aadcb2b0915b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:32.379616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fcd85-d4ea-4bbe-9441-aadcb2b0915b None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:32.380115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fcd85-d4ea-4bbe-9441-aadcb2b0915b None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:32.380864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fcd85-d4ea-4bbe-9441-aadcb2b0915b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:32.395876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fcd85-d4ea-4bbe-9441-aadcb2b0915b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-1d1fcd85-d4ea-4bbe-9441-aadcb2b0915b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:32.405099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:32.406066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:32.406880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:32.406880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:32.406880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:32.406880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:32.406880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:32.406880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:32.406880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:32.406880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:32.406880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:32.406880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:32.406880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:32.406880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:32.410501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 170/344] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:54:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:32.412380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c649252-ba46-493f-8eb8-62f914fc7a0b None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:32.412860 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c649252-ba46-493f-8eb8-62f914fc7a0b None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:32.413322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c649252-ba46-493f-8eb8-62f914fc7a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:32.423794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a9ee3-bb36-41bb-bf76-5d83bef9e540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:32.429652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a9ee3-bb36-41bb-bf76-5d83bef9e540 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:32.430164 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a9ee3-bb36-41bb-bf76-5d83bef9e540 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:32.430650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a9ee3-bb36-41bb-bf76-5d83bef9e540 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:32.444220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-9c649252-ba46-493f-8eb8-62f914fc7a0b None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:32.447965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-9c649252-ba46-493f-8eb8-62f914fc7a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:32.503826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:32.540308 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a9ee3-bb36-41bb-bf76-5d83bef9e540 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:32.542940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796a9ee3-bb36-41bb-bf76-5d83bef9e540 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:32.548896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 171/345] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:32.555610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c649252-ba46-493f-8eb8-62f914fc7a0b None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:32.563795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:32.654733 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 175/346] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:32.661104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a9ee3-bb36-41bb-bf76-5d83bef9e540 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:32.729038 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c649252-ba46-493f-8eb8-62f914fc7a0b None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:32.769256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267a6b0-734e-435f-9b93-ca97f55589e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:32.769879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267a6b0-734e-435f-9b93-ca97f55589e4 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:32.770155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267a6b0-734e-435f-9b93-ca97f55589e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:32.770424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267a6b0-734e-435f-9b93-ca97f55589e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:32.771109 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c267a6b0-734e-435f-9b93-ca97f55589e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:32.771407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c267a6b0-734e-435f-9b93-ca97f55589e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:32.772842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c267a6b0-734e-435f-9b93-ca97f55589e4 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:32.787558 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 172/347] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:32.788475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267a6b0-734e-435f-9b93-ca97f55589e4 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:32.845855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:32.867062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796a9ee3-bb36-41bb-bf76-5d83bef9e540 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:32.897386 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:32.925516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22889e0-5dff-4bb8-b5e2-f03974a5b1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:32.926080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22889e0-5dff-4bb8-b5e2-f03974a5b1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:32.926357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22889e0-5dff-4bb8-b5e2-f03974a5b1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:32.926627 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22889e0-5dff-4bb8-b5e2-f03974a5b1d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:32.938770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22889e0-5dff-4bb8-b5e2-f03974a5b1d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-c22889e0-5dff-4bb8-b5e2-f03974a5b1d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:32.945410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:32.946404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:32.947231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:32.947231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:32.947231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:32.947231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:32.947231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:32.947231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:32.947231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:32.947231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:32.947231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:32.947231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:32.947231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 15:54:32.947231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:32.947231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 176/348] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:32.947941 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22889e0-5dff-4bb8-b5e2-f03974a5b1d0 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:32.983196 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267a6b0-734e-435f-9b93-ca97f55589e4 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:33.023048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd98a3b-a563-4794-9730-5cfaaecf4a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:33.024171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd98a3b-a563-4794-9730-5cfaaecf4a03 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:33.024645 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd98a3b-a563-4794-9730-5cfaaecf4a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:33.025109 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd98a3b-a563-4794-9730-5cfaaecf4a03 None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:33.026497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd98a3b-a563-4794-9730-5cfaaecf4a03 None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:33.028179 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd98a3b-a563-4794-9730-5cfaaecf4a03 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:33.035330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 173/349] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:32 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:33.036527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd98a3b-a563-4794-9730-5cfaaecf4a03 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:33.074761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22889e0-5dff-4bb8-b5e2-f03974a5b1d0 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:33.117384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5167e8-f9ec-422e-bfa6-677f42eee830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:33.118008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5167e8-f9ec-422e-bfa6-677f42eee830 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:33.118287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5167e8-f9ec-422e-bfa6-677f42eee830 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:33.118561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5167e8-f9ec-422e-bfa6-677f42eee830 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:33.148973 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:33.156162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd98a3b-a563-4794-9730-5cfaaecf4a03 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:33.199675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac22fc-1d0b-411b-a776-21094005c454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:33.200636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac22fc-1d0b-411b-a776-21094005c454 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:33.201108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac22fc-1d0b-411b-a776-21094005c454 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:33.201597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac22fc-1d0b-411b-a776-21094005c454 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:33.202487 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac22fc-1d0b-411b-a776-21094005c454 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:33.202985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac22fc-1d0b-411b-a776-21094005c454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:33.204673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bac22fc-1d0b-411b-a776-21094005c454 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:33.212038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5167e8-f9ec-422e-bfa6-677f42eee830 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:33.214098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5167e8-f9ec-422e-bfa6-677f42eee830 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:33.215438 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 174/350] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:33.215438 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac22fc-1d0b-411b-a776-21094005c454 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:33.223793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:33.351005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 177/351] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:33.352355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5167e8-f9ec-422e-bfa6-677f42eee830 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:33.363086 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac22fc-1d0b-411b-a776-21094005c454 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:33.406762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891f0ba-aed3-4be2-84e1-3e52f4a9bb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:33.407973 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891f0ba-aed3-4be2-84e1-3e52f4a9bb6b None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:33.407973 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891f0ba-aed3-4be2-84e1-3e52f4a9bb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:33.408162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f891f0ba-aed3-4be2-84e1-3e52f4a9bb6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:33.424338 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f891f0ba-aed3-4be2-84e1-3e52f4a9bb6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-f891f0ba-aed3-4be2-84e1-3e52f4a9bb6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:33.430313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:33.431228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:33.432081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:33.432081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:33.432081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:33.432081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:33.432081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:33.432081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:33.432081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:33.432081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:33.432081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:33.432081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:33.432081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:33.432081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:33.432690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 175/352] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:33.435296 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891f0ba-aed3-4be2-84e1-3e52f4a9bb6b None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:33.470407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:33.478473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:33.506307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5167e8-f9ec-422e-bfa6-677f42eee830 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:33.546026 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef299fa-826c-4107-a7f0-c1312f8b4b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:33.546674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef299fa-826c-4107-a7f0-c1312f8b4b43 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:33.547122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef299fa-826c-4107-a7f0-c1312f8b4b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:33.547224 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef299fa-826c-4107-a7f0-c1312f8b4b43 None admin] PATH_INFO: `/v3/groups/d08827f6950a4a2999950a5d803b58ab` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:33.553827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef299fa-826c-4107-a7f0-c1312f8b4b43 None admin] RBAC: Authorizing `identity:get_group(group_id=d08827f6950a4a2999950a5d803b58ab)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:33.555642 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef299fa-826c-4107-a7f0-c1312f8b4b43 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:33.557295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 178/353] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Sep 8 15:54:33 2022] GET /identity/v3/groups/d08827f6950a4a2999950a5d803b58ab => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:33.558517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef299fa-826c-4107-a7f0-c1312f8b4b43 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:33.570894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f891f0ba-aed3-4be2-84e1-3e52f4a9bb6b None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:33.612738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ae8cb-e0af-4bad-9b80-0fbe0c55e8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:33.613389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ae8cb-e0af-4bad-9b80-0fbe0c55e8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:33.613669 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ae8cb-e0af-4bad-9b80-0fbe0c55e8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:33.613943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ae8cb-e0af-4bad-9b80-0fbe0c55e8a5 None admin] PATH_INFO: `/v3/users/03a5d01f5f104dbcaa197ee7900b81d0` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:33.615094 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ae8cb-e0af-4bad-9b80-0fbe0c55e8a5 None admin] RBAC: Authorizing `identity:get_user(user_id=03a5d01f5f104dbcaa197ee7900b81d0)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:33.616891 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ae8cb-e0af-4bad-9b80-0fbe0c55e8a5 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:33.618742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 176/354] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:54:33 2022] GET /identity/v3/users/03a5d01f5f104dbcaa197ee7900b81d0 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:33.619302 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ae8cb-e0af-4bad-9b80-0fbe0c55e8a5 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:33.681370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef299fa-826c-4107-a7f0-c1312f8b4b43 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:33.720613 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:33.725084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f4be3f-97ce-49ab-b9fc-b47116a9b3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:33.725591 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4be3f-97ce-49ab-b9fc-b47116a9b3ca None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:33.725848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4be3f-97ce-49ab-b9fc-b47116a9b3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:33.726115 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f4be3f-97ce-49ab-b9fc-b47116a9b3ca None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:33.727392 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4be3f-97ce-49ab-b9fc-b47116a9b3ca None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:33.729202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f4be3f-97ce-49ab-b9fc-b47116a9b3ca None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:33.731869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 179/355] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:33 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:33.733355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf245a0-841d-486c-9766-9299e53f9374 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:33.733643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf245a0-841d-486c-9766-9299e53f9374 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:33.733915 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf245a0-841d-486c-9766-9299e53f9374 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:33.735282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 180/356] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:33.736865 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ae8cb-e0af-4bad-9b80-0fbe0c55e8a5 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:33.741723 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd777731-93b3-4e82-bbb2-47803174eef5 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:33.742003 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd777731-93b3-4e82-bbb2-47803174eef5 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:33.742278 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd777731-93b3-4e82-bbb2-47803174eef5 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:33.743645 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 181/357] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:33.744505 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd777731-93b3-4e82-bbb2-47803174eef5 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:33.758150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:33.778135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fec8f-d3a2-492b-96ea-db3f02cbd3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:33.778898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fec8f-d3a2-492b-96ea-db3f02cbd3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:33.779171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fec8f-d3a2-492b-96ea-db3f02cbd3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:33.780348 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fec8f-d3a2-492b-96ea-db3f02cbd3a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:33.780564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fec8f-d3a2-492b-96ea-db3f02cbd3a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:33.780868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fec8f-d3a2-492b-96ea-db3f02cbd3a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:33.782388 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4fec8f-d3a2-492b-96ea-db3f02cbd3a2 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:33.793210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 177/358] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:33.794042 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fec8f-d3a2-492b-96ea-db3f02cbd3a2 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:33.862331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd777731-93b3-4e82-bbb2-47803174eef5 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:33.901994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5dc88a-8244-4594-afab-06b492e8db15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:33.902607 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5dc88a-8244-4594-afab-06b492e8db15 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:33.902881 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5dc88a-8244-4594-afab-06b492e8db15 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:33.903149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5dc88a-8244-4594-afab-06b492e8db15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:33.904122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5dc88a-8244-4594-afab-06b492e8db15 None admin] RBAC: Adding query filter params (group.id=d08827f6950a4a2999950a5d803b58ab, role.id=abbd88ef58a544db8ee4261548778980, scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:33.904360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5dc88a-8244-4594-afab-06b492e8db15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:33.905805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5dc88a-8244-4594-afab-06b492e8db15 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:33.911875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 182/359] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Sep 8 15:54:33 2022] GET /identity/v3/role_assignments?group.id=d08827f6950a4a2999950a5d803b58ab&scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e&role.id=abbd88ef58a544db8ee4261548778980 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:33.912669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5dc88a-8244-4594-afab-06b492e8db15 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:33.919020 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fec8f-d3a2-492b-96ea-db3f02cbd3a2 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:33.966575 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b6d7d-330a-4ac0-bf8f-863b2b2bfa05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:33.968261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b6d7d-330a-4ac0-bf8f-863b2b2bfa05 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:33.968743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b6d7d-330a-4ac0-bf8f-863b2b2bfa05 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:33.969209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b6d7d-330a-4ac0-bf8f-863b2b2bfa05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:33.970566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b6d7d-330a-4ac0-bf8f-863b2b2bfa05 None admin] RBAC: Adding query filter params (role.id=a9c50f8878284257bb064b71a6f7870b, user.id=03a5d01f5f104dbcaa197ee7900b81d0, scope.system=all) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:33.970931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b6d7d-330a-4ac0-bf8f-863b2b2bfa05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:33.973118 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b6d7d-330a-4ac0-bf8f-863b2b2bfa05 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:33.989419 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 178/360] 10.222.0.27 () {56 vars in 1308 bytes} [Thu Sep 8 15:54:33 2022] GET /identity/v3/role_assignments?user.id=03a5d01f5f104dbcaa197ee7900b81d0&scope.system=all&role.id=a9c50f8878284257bb064b71a6f7870b => generated 510 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:33.990710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c31c03-7544-4872-ab62-5ddb5c9f6565 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:33.990992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c31c03-7544-4872-ab62-5ddb5c9f6565 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:33.991264 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c31c03-7544-4872-ab62-5ddb5c9f6565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:34.012670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:34.019785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-65c31c03-7544-4872-ab62-5ddb5c9f6565 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:34.020961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-65c31c03-7544-4872-ab62-5ddb5c9f6565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:34.037053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:34.059683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5dc88a-8244-4594-afab-06b492e8db15 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:34.115666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973d766-4257-4276-82be-e6489d9eac06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:34.116354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973d766-4257-4276-82be-e6489d9eac06 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:34.116708 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973d766-4257-4276-82be-e6489d9eac06 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:34.116984 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973d766-4257-4276-82be-e6489d9eac06 None admin] PATH_INFO: `/v3/groups/d08827f6950a4a2999950a5d803b58ab` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:34.118190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973d766-4257-4276-82be-e6489d9eac06 None admin] RBAC: Authorizing `identity:get_group(group_id=d08827f6950a4a2999950a5d803b58ab)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:34.118314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 179/361] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:34.120207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e523e-396d-4e51-96f5-efe72ec067e6 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:34.120327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973d766-4257-4276-82be-e6489d9eac06 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:34.120402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e523e-396d-4e51-96f5-efe72ec067e6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:34.120402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e523e-396d-4e51-96f5-efe72ec067e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:34.121615 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 183/362] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Sep 8 15:54:33 2022] GET /identity/v3/groups/d08827f6950a4a2999950a5d803b58ab => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:34.122400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973d766-4257-4276-82be-e6489d9eac06 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:34.142670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-0c4e523e-396d-4e51-96f5-efe72ec067e6 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:34.143568 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0c4e523e-396d-4e51-96f5-efe72ec067e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:34.262756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 180/363] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:34.264492 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e523e-396d-4e51-96f5-efe72ec067e6 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:34.273761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973d766-4257-4276-82be-e6489d9eac06 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:34.337221 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0aa58-b20a-405c-a720-4daae9132575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:34.338567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0aa58-b20a-405c-a720-4daae9132575 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:34.339056 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0aa58-b20a-405c-a720-4daae9132575 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:34.339558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0aa58-b20a-405c-a720-4daae9132575 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:34.357768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:34.372796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:34.439524 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e523e-396d-4e51-96f5-efe72ec067e6 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:34.455856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0aa58-b20a-405c-a720-4daae9132575 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:34.457411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff0aa58-b20a-405c-a720-4daae9132575 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:34.507407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64566628-1c3f-42bf-8857-98d6d5f6f540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:34.508445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64566628-1c3f-42bf-8857-98d6d5f6f540 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:34.509255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64566628-1c3f-42bf-8857-98d6d5f6f540 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:34.509720 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64566628-1c3f-42bf-8857-98d6d5f6f540 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:34.511334 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64566628-1c3f-42bf-8857-98d6d5f6f540 None admin] RBAC: Adding query filter params (group.id=d08827f6950a4a2999950a5d803b58ab, role.id=bf98c97818504bc48e3bd1b2aef9d3c3, scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:34.512498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64566628-1c3f-42bf-8857-98d6d5f6f540 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:34.514373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64566628-1c3f-42bf-8857-98d6d5f6f540 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:34.528802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 181/364] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Sep 8 15:54:34 2022] GET /identity/v3/role_assignments?group.id=d08827f6950a4a2999950a5d803b58ab&scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e&role.id=bf98c97818504bc48e3bd1b2aef9d3c3 => generated 260 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:34.530098 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64566628-1c3f-42bf-8857-98d6d5f6f540 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:34.612075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 184/365] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:34.612075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0aa58-b20a-405c-a720-4daae9132575 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:34.668583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64566628-1c3f-42bf-8857-98d6d5f6f540 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:34.674753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:34.707855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:34.722556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc9899-650b-456c-a486-4e30944eb99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:34.722907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc9899-650b-456c-a486-4e30944eb99d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:34.723160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc9899-650b-456c-a486-4e30944eb99d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:34.723435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc9899-650b-456c-a486-4e30944eb99d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:34.739416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc9899-650b-456c-a486-4e30944eb99d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-79cc9899-650b-456c-a486-4e30944eb99d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:34.747529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:34.748757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:34.749817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:34.749817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:34.749817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:34.749817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:34.749817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:34.749817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:34.749817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:34.749817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:34.749817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:34.749817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:34.749817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:54:34.749817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:34.750839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 182/366] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:54:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:34.752176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc9899-650b-456c-a486-4e30944eb99d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:34.755192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0aa58-b20a-405c-a720-4daae9132575 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:34.800213 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b524e9-c430-4b6a-b41f-6133658162b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:34.801236 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b524e9-c430-4b6a-b41f-6133658162b2 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:34.801758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b524e9-c430-4b6a-b41f-6133658162b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:34.802220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b524e9-c430-4b6a-b41f-6133658162b2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:34.813643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b524e9-c430-4b6a-b41f-6133658162b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-84b524e9-c430-4b6a-b41f-6133658162b2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:34.818936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:34.820043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:34.820838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:34.820838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:34.820838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:34.820838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:34.820838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:34.820838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:34.820838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:34.820838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:34.820838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:34.820838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:34.820838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:34.820838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:34.826886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 185/367] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:54:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:34.826886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b524e9-c430-4b6a-b41f-6133658162b2 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:34.876994 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc9899-650b-456c-a486-4e30944eb99d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:34.914215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9291b98-5ec5-4906-9676-a4b3412e9068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:34.914866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9291b98-5ec5-4906-9676-a4b3412e9068 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:34.915140 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9291b98-5ec5-4906-9676-a4b3412e9068 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:34.915409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9291b98-5ec5-4906-9676-a4b3412e9068 None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:34.916596 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9291b98-5ec5-4906-9676-a4b3412e9068 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:34.918059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9291b98-5ec5-4906-9676-a4b3412e9068 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:34.923571 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 183/368] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:34 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:34.923571 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9291b98-5ec5-4906-9676-a4b3412e9068 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:34.954603 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:34.961461 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b524e9-c430-4b6a-b41f-6133658162b2 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:34.996750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:35.011092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b366d-60a3-4a94-b25f-a553104fb513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:35.011780 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b366d-60a3-4a94-b25f-a553104fb513 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:35.012057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b366d-60a3-4a94-b25f-a553104fb513 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:35.012326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b366d-60a3-4a94-b25f-a553104fb513 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:35.013001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b366d-60a3-4a94-b25f-a553104fb513 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:35.013300 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b366d-60a3-4a94-b25f-a553104fb513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:35.014789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b366d-60a3-4a94-b25f-a553104fb513 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:35.026331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 186/369] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:35.032440 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b366d-60a3-4a94-b25f-a553104fb513 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:35.054556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9291b98-5ec5-4906-9676-a4b3412e9068 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:35.106311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb4ea4-1ba3-4460-a56e-69f7b1efab80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:35.107038 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb4ea4-1ba3-4460-a56e-69f7b1efab80 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:35.107305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb4ea4-1ba3-4460-a56e-69f7b1efab80 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:35.108101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb4ea4-1ba3-4460-a56e-69f7b1efab80 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:35.108776 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb4ea4-1ba3-4460-a56e-69f7b1efab80 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:35.109075 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb4ea4-1ba3-4460-a56e-69f7b1efab80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:35.110615 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facb4ea4-1ba3-4460-a56e-69f7b1efab80 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:35.120692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 184/370] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:35.121700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb4ea4-1ba3-4460-a56e-69f7b1efab80 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:35.202750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b366d-60a3-4a94-b25f-a553104fb513 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:35.239941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb4ea4-1ba3-4460-a56e-69f7b1efab80 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:35.246171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9542c-f9db-415b-8e02-5285e15a507b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:35.246822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9542c-f9db-415b-8e02-5285e15a507b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:35.247099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9542c-f9db-415b-8e02-5285e15a507b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:35.247952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9542c-f9db-415b-8e02-5285e15a507b None admin] PATH_INFO: `/v3/groups/d08827f6950a4a2999950a5d803b58ab` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:35.249108 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9542c-f9db-415b-8e02-5285e15a507b None admin] RBAC: Authorizing `identity:get_group(group_id=d08827f6950a4a2999950a5d803b58ab)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:35.251050 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9542c-f9db-415b-8e02-5285e15a507b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:35.253466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 187/371] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Sep 8 15:54:35 2022] GET /identity/v3/groups/d08827f6950a4a2999950a5d803b58ab => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:35.254244 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9542c-f9db-415b-8e02-5285e15a507b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:35.267697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:35.289228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9ad9b-3a9f-4313-8524-19dd642fce4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:35.290122 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9ad9b-3a9f-4313-8524-19dd642fce4a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:35.290556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9ad9b-3a9f-4313-8524-19dd642fce4a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:35.290963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c9ad9b-3a9f-4313-8524-19dd642fce4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:35.301447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:35.378115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9ad9b-3a9f-4313-8524-19dd642fce4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:35.380078 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c9ad9b-3a9f-4313-8524-19dd642fce4a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:35.381024 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9542c-f9db-415b-8e02-5285e15a507b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:35.419198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9595e-bb98-442a-800f-c6c38b9cc657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:35.419849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9595e-bb98-442a-800f-c6c38b9cc657 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:35.420123 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9595e-bb98-442a-800f-c6c38b9cc657 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:35.420503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9595e-bb98-442a-800f-c6c38b9cc657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:35.524720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9595e-bb98-442a-800f-c6c38b9cc657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:35.527106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9595e-bb98-442a-800f-c6c38b9cc657 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:35.534959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 185/372] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:35.535972 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9ad9b-3a9f-4313-8524-19dd642fce4a None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:35.558275 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:35.562249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:35.661640 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c9ad9b-3a9f-4313-8524-19dd642fce4a None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:35.666964 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 188/373] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:35.668032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9595e-bb98-442a-800f-c6c38b9cc657 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:35.703921 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dff96-2309-48ba-b41e-004021057a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:35.704535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dff96-2309-48ba-b41e-004021057a87 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:35.704812 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dff96-2309-48ba-b41e-004021057a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:35.705070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dff96-2309-48ba-b41e-004021057a87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:35.705797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161dff96-2309-48ba-b41e-004021057a87 None admin] RBAC: Adding query filter params (group.id=d08827f6950a4a2999950a5d803b58ab, role.id=abbd88ef58a544db8ee4261548778980, scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:35.706135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161dff96-2309-48ba-b41e-004021057a87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:35.707692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161dff96-2309-48ba-b41e-004021057a87 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:35.714414 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 186/374] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Sep 8 15:54:35 2022] GET /identity/v3/role_assignments?group.id=d08827f6950a4a2999950a5d803b58ab&scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd&role.id=abbd88ef58a544db8ee4261548778980 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:35.715225 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dff96-2309-48ba-b41e-004021057a87 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:35.794770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9595e-bb98-442a-800f-c6c38b9cc657 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:35.829399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:35.834968 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:35.838034 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181727e-ca72-4676-a3a3-015d4b99760c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:35.839305 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181727e-ca72-4676-a3a3-015d4b99760c None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:35.840017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181727e-ca72-4676-a3a3-015d4b99760c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:35.840387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181727e-ca72-4676-a3a3-015d4b99760c None admin] PATH_INFO: `/v3/projects/e21b84c9f53a40aa98d4a318db545714` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:35.841883 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181727e-ca72-4676-a3a3-015d4b99760c None admin] RBAC: Authorizing `identity:get_project(project_id=e21b84c9f53a40aa98d4a318db545714)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:35.843744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181727e-ca72-4676-a3a3-015d4b99760c None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:35.846048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 189/375] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:35 2022] GET /identity/v3/projects/e21b84c9f53a40aa98d4a318db545714 => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:35.851994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181727e-ca72-4676-a3a3-015d4b99760c None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:35.858868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dff96-2309-48ba-b41e-004021057a87 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:35.906651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a188e40-daba-443e-9730-5472fb8a60e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:35.907292 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a188e40-daba-443e-9730-5472fb8a60e5 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:35.907705 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a188e40-daba-443e-9730-5472fb8a60e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:35.908072 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a188e40-daba-443e-9730-5472fb8a60e5 None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:35.909449 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a188e40-daba-443e-9730-5472fb8a60e5 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:35.911101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a188e40-daba-443e-9730-5472fb8a60e5 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:35.913197 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 187/376] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:35 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:35.918730 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a188e40-daba-443e-9730-5472fb8a60e5 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:35.986349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181727e-ca72-4676-a3a3-015d4b99760c None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:36.030446 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535d968-8be0-4598-be1c-56fc210f32fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:36.031295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535d968-8be0-4598-be1c-56fc210f32fd None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:36.031714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535d968-8be0-4598-be1c-56fc210f32fd None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:36.032083 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535d968-8be0-4598-be1c-56fc210f32fd None admin] PATH_INFO: `/v3/groups/66861b522a6b4058bce322d39977662b` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:36.045701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535d968-8be0-4598-be1c-56fc210f32fd None admin] RBAC: Authorizing `identity:get_group(group_id=66861b522a6b4058bce322d39977662b)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:36.047231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535d968-8be0-4598-be1c-56fc210f32fd None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:36.050900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 190/377] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Sep 8 15:54:35 2022] GET /identity/v3/groups/66861b522a6b4058bce322d39977662b => generated 229 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:36.066288 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a188e40-daba-443e-9730-5472fb8a60e5 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:36.073607 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535d968-8be0-4598-be1c-56fc210f32fd None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:36.110178 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd7916-c621-4955-8aac-1c24cc79077e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:36.110835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd7916-c621-4955-8aac-1c24cc79077e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:36.111109 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd7916-c621-4955-8aac-1c24cc79077e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:36.111380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cd7916-c621-4955-8aac-1c24cc79077e None admin] PATH_INFO: `/v3/groups/d08827f6950a4a2999950a5d803b58ab` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:36.113498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd7916-c621-4955-8aac-1c24cc79077e None admin] RBAC: Authorizing `identity:get_group(group_id=d08827f6950a4a2999950a5d803b58ab)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:36.115066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cd7916-c621-4955-8aac-1c24cc79077e None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:36.117246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 188/378] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Sep 8 15:54:35 2022] GET /identity/v3/groups/d08827f6950a4a2999950a5d803b58ab => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:36.123158 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd7916-c621-4955-8aac-1c24cc79077e None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:36.149157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:36.179911 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:36.201562 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535d968-8be0-4598-be1c-56fc210f32fd None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:36.240884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff5525-572a-4f9f-8e6c-eaa7a439f794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:36.241513 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff5525-572a-4f9f-8e6c-eaa7a439f794 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:36.241768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff5525-572a-4f9f-8e6c-eaa7a439f794 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:36.242040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff5525-572a-4f9f-8e6c-eaa7a439f794 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:36.243240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff5525-572a-4f9f-8e6c-eaa7a439f794 None admin] RBAC: Adding query filter params (group.id=66861b522a6b4058bce322d39977662b, role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=e21b84c9f53a40aa98d4a318db545714) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:36.243406 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff5525-572a-4f9f-8e6c-eaa7a439f794 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:36.245154 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff5525-572a-4f9f-8e6c-eaa7a439f794 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:36.252250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 191/379] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Sep 8 15:54:36 2022] GET /identity/v3/role_assignments?group.id=66861b522a6b4058bce322d39977662b&scope.project.id=e21b84c9f53a40aa98d4a318db545714&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:36.270129 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cd7916-c621-4955-8aac-1c24cc79077e None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:36.334145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a979c08-1eb5-4adc-91e6-e84e762ea9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:36.335052 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a979c08-1eb5-4adc-91e6-e84e762ea9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:36.335501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a979c08-1eb5-4adc-91e6-e84e762ea9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:36.335932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a979c08-1eb5-4adc-91e6-e84e762ea9c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:36.336812 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a979c08-1eb5-4adc-91e6-e84e762ea9c1 None admin] RBAC: Adding query filter params (group.id=d08827f6950a4a2999950a5d803b58ab, role.id=bf98c97818504bc48e3bd1b2aef9d3c3, scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:36.337286 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a979c08-1eb5-4adc-91e6-e84e762ea9c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:36.339268 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a979c08-1eb5-4adc-91e6-e84e762ea9c1 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:36.345853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 189/380] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Sep 8 15:54:36 2022] GET /identity/v3/role_assignments?group.id=d08827f6950a4a2999950a5d803b58ab&scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd&role.id=bf98c97818504bc48e3bd1b2aef9d3c3 => generated 260 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:36.693256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51117cea-f449-4060-8ed6-3ca8bebb2f9b None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:36.693660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51117cea-f449-4060-8ed6-3ca8bebb2f9b None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:36.693891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51117cea-f449-4060-8ed6-3ca8bebb2f9b None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:36.695515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 192/381] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:36.701244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f976b-6484-427a-8659-0463e3dff2b0 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:36.701530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f976b-6484-427a-8659-0463e3dff2b0 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:36.701813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0f976b-6484-427a-8659-0463e3dff2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:36.723769 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-ca0f976b-6484-427a-8659-0463e3dff2b0 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:36.725034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ca0f976b-6484-427a-8659-0463e3dff2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:36.822448 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 190/382] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:37.138182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0d4d2-7b6e-45c6-9e01-bf8b3685a746 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:37.138582 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0d4d2-7b6e-45c6-9e01-bf8b3685a746 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:37.138795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0d4d2-7b6e-45c6-9e01-bf8b3685a746 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:37.140461 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 193/383] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:37.141487 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f976b-6484-427a-8659-0463e3dff2b0 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:37.156726 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa3c4c-090b-4f78-b0ce-0da83ed0a4fd None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:37.157042 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa3c4c-090b-4f78-b0ce-0da83ed0a4fd None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:37.157275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aa3c4c-090b-4f78-b0ce-0da83ed0a4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:37.160965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:37.185936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-f4aa3c4c-090b-4f78-b0ce-0da83ed0a4fd None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:37.187143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f4aa3c4c-090b-4f78-b0ce-0da83ed0a4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:37.305158 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 194/384] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:37.313267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0f976b-6484-427a-8659-0463e3dff2b0 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:37.350411 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd62cd9-e0df-4fda-82f1-1374e14e112a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:37.351056 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd62cd9-e0df-4fda-82f1-1374e14e112a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:37.351377 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd62cd9-e0df-4fda-82f1-1374e14e112a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:37.351697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd62cd9-e0df-4fda-82f1-1374e14e112a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:37.363576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd62cd9-e0df-4fda-82f1-1374e14e112a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-fbd62cd9-e0df-4fda-82f1-1374e14e112a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:37.368489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:37.369361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:37.370333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:37.370333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:37.370333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:37.370333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:37.370333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:37.370333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:37.370333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:37.370333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:37.370333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:37.370333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:37.370333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:37.370333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:37.370333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 191/385] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:37.376057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa3c4c-090b-4f78-b0ce-0da83ed0a4fd None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:37.448169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:37.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aa3c4c-090b-4f78-b0ce-0da83ed0a4fd None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:37.543897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49533618-2057-4587-80f7-aec25ea8ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:37.544618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49533618-2057-4587-80f7-aec25ea8ec09 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:37.544893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49533618-2057-4587-80f7-aec25ea8ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:37.545161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49533618-2057-4587-80f7-aec25ea8ec09 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:37.545859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49533618-2057-4587-80f7-aec25ea8ec09 None admin] RBAC: Adding query filter params (name=member) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:37.546159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49533618-2057-4587-80f7-aec25ea8ec09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:37.547686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49533618-2057-4587-80f7-aec25ea8ec09 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:37.557610 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 195/386] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:37.564553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd62cd9-e0df-4fda-82f1-1374e14e112a None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:37.645512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:37.676239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd62cd9-e0df-4fda-82f1-1374e14e112a None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:37.706278 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49533618-2057-4587-80f7-aec25ea8ec09 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:37.722747 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f35df-87d8-46bd-9f50-0dc391066dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:37.723408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f35df-87d8-46bd-9f50-0dc391066dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:37.723704 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f35df-87d8-46bd-9f50-0dc391066dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:37.723969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695f35df-87d8-46bd-9f50-0dc391066dd0 None admin] PATH_INFO: `/v3/groups/d08827f6950a4a2999950a5d803b58ab` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:37.728342 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f35df-87d8-46bd-9f50-0dc391066dd0 None admin] RBAC: Authorizing `identity:get_group(group_id=d08827f6950a4a2999950a5d803b58ab)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:37.730081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695f35df-87d8-46bd-9f50-0dc391066dd0 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:37.732220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 192/387] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Sep 8 15:54:37 2022] GET /identity/v3/groups/d08827f6950a4a2999950a5d803b58ab => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:37.739297 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f35df-87d8-46bd-9f50-0dc391066dd0 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:37.851496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49533618-2057-4587-80f7-aec25ea8ec09 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:37.879470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:37.885114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695f35df-87d8-46bd-9f50-0dc391066dd0 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:37.904145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee91b6-8653-41b4-9484-a3b1eb69f8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:37.905147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee91b6-8653-41b4-9484-a3b1eb69f8a8 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:37.905620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee91b6-8653-41b4-9484-a3b1eb69f8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:37.906076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee91b6-8653-41b4-9484-a3b1eb69f8a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:37.916899 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee91b6-8653-41b4-9484-a3b1eb69f8a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-58ee91b6-8653-41b4-9484-a3b1eb69f8a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:37.922489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:37.923357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:37.924181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:37.924181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:37.924181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:37.924181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:37.924181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:37.924181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:37.924181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:37.924181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:37.924181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:37.924181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:37.924181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:37.924181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:37.926898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 196/388] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:54:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:54:37.932500 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee91b6-8653-41b4-9484-a3b1eb69f8a8 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:37.953482 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51817d-cd40-4e4b-8bea-d6686b449597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:37.954132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51817d-cd40-4e4b-8bea-d6686b449597 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:37.954415 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51817d-cd40-4e4b-8bea-d6686b449597 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:37.954686 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51817d-cd40-4e4b-8bea-d6686b449597 None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:37.960269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51817d-cd40-4e4b-8bea-d6686b449597 None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:37.961567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51817d-cd40-4e4b-8bea-d6686b449597 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:37.964087 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 193/389] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:37 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:37.969716 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51817d-cd40-4e4b-8bea-d6686b449597 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:37.995094 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:38.040492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee91b6-8653-41b4-9484-a3b1eb69f8a8 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:38.087409 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c9a21-e18d-4af4-8a8f-97bc1afb330a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:38.088402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c9a21-e18d-4af4-8a8f-97bc1afb330a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:38.088681 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c9a21-e18d-4af4-8a8f-97bc1afb330a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:38.088952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c9a21-e18d-4af4-8a8f-97bc1afb330a None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:38.089634 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c9a21-e18d-4af4-8a8f-97bc1afb330a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:38.089938 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c9a21-e18d-4af4-8a8f-97bc1afb330a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:38.094921 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c9a21-e18d-4af4-8a8f-97bc1afb330a None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:38.112042 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 197/390] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:38.118212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c9a21-e18d-4af4-8a8f-97bc1afb330a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:38.144430 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51817d-cd40-4e4b-8bea-d6686b449597 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:38.196939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af263fc-41d0-48fe-9fb4-2f457f4c31fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:38.197912 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af263fc-41d0-48fe-9fb4-2f457f4c31fc None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:38.198523 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af263fc-41d0-48fe-9fb4-2f457f4c31fc None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:38.199405 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af263fc-41d0-48fe-9fb4-2f457f4c31fc None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e/groups/d08827f6950a4a2999950a5d803b58ab/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:38.201507 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af263fc-41d0-48fe-9fb4-2f457f4c31fc None admin] RBAC: Authorizing `identity:create_grant(project_id=ae75fe116ff145cdb11b0b2127b2ff7e, group_id=d08827f6950a4a2999950a5d803b58ab, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:38.203176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af263fc-41d0-48fe-9fb4-2f457f4c31fc None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:38.222585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 194/391] 10.222.0.27 () {58 vars in 1442 bytes} [Thu Sep 8 15:54:37 2022] PUT /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e/groups/d08827f6950a4a2999950a5d803b58ab/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 15:54:38.230522 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:38.331886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c9a21-e18d-4af4-8a8f-97bc1afb330a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:38.389323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f52e0-9dd4-4776-9177-9286964b62f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:38.389996 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f52e0-9dd4-4776-9177-9286964b62f2 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:38.390276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f52e0-9dd4-4776-9177-9286964b62f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:38.390549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192f52e0-9dd4-4776-9177-9286964b62f2 None admin] PATH_INFO: `/v3/groups/d08827f6950a4a2999950a5d803b58ab` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:38.391748 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f52e0-9dd4-4776-9177-9286964b62f2 None admin] RBAC: Authorizing `identity:get_group(group_id=d08827f6950a4a2999950a5d803b58ab)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:38.393255 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192f52e0-9dd4-4776-9177-9286964b62f2 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:38.395032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 198/392] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Sep 8 15:54:38 2022] GET /identity/v3/groups/d08827f6950a4a2999950a5d803b58ab => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:38.400801 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af263fc-41d0-48fe-9fb4-2f457f4c31fc None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:38.508169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:38.553954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af263fc-41d0-48fe-9fb4-2f457f4c31fc None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:38.597373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2511d-6cb3-44e4-9645-494801cd2fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:38.598083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2511d-6cb3-44e4-9645-494801cd2fbd None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:38.598357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2511d-6cb3-44e4-9645-494801cd2fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:38.598637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2511d-6cb3-44e4-9645-494801cd2fbd None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:38.603217 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2511d-6cb3-44e4-9645-494801cd2fbd None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:38.604763 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c2511d-6cb3-44e4-9645-494801cd2fbd None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:38.606668 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 195/393] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:38 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:38.612921 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f52e0-9dd4-4776-9177-9286964b62f2 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:38.724413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c24067-8d1f-4857-b309-ffbc7d67cce9 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:38.724774 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c24067-8d1f-4857-b309-ffbc7d67cce9 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:38.724966 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c24067-8d1f-4857-b309-ffbc7d67cce9 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:38.726570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 196/394] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:38.731655 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28777f7b-756c-4422-93ad-ed944432ee32 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:38.731925 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28777f7b-756c-4422-93ad-ed944432ee32 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:38.732195 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28777f7b-756c-4422-93ad-ed944432ee32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:38.750929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-28777f7b-756c-4422-93ad-ed944432ee32 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:38.752259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-28777f7b-756c-4422-93ad-ed944432ee32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:38.755813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192f52e0-9dd4-4776-9177-9286964b62f2 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:38.771891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:38.813584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8c054-60e7-4bf7-b858-c082cad16a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:38.814272 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8c054-60e7-4bf7-b858-c082cad16a6a None admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:38.814551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8c054-60e7-4bf7-b858-c082cad16a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:38.814822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8c054-60e7-4bf7-b858-c082cad16a6a None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e/groups/d08827f6950a4a2999950a5d803b58ab/roles/bf98c97818504bc48e3bd1b2aef9d3c3` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:38.822307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8c054-60e7-4bf7-b858-c082cad16a6a None admin] RBAC: Authorizing `identity:create_grant(project_id=ae75fe116ff145cdb11b0b2127b2ff7e, group_id=d08827f6950a4a2999950a5d803b58ab, role_id=bf98c97818504bc48e3bd1b2aef9d3c3)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:38.823800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8c054-60e7-4bf7-b858-c082cad16a6a None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:38.850790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 199/395] 10.222.0.27 () {58 vars in 1442 bytes} [Thu Sep 8 15:54:38 2022] PUT /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e/groups/d08827f6950a4a2999950a5d803b58ab/roles/bf98c97818504bc48e3bd1b2aef9d3c3 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:54:38.864976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 197/396] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:39.183489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11015ac-3037-495f-bdb3-e6cdd00131b9 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:39.183875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11015ac-3037-495f-bdb3-e6cdd00131b9 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:39.184077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11015ac-3037-495f-bdb3-e6cdd00131b9 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:39.185658 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 200/397] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:39.191161 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae79407-cab4-4cac-8ebb-27efa64a1bde None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:39.191476 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae79407-cab4-4cac-8ebb-27efa64a1bde None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:39.191755 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae79407-cab4-4cac-8ebb-27efa64a1bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:39.220450 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-9ae79407-cab4-4cac-8ebb-27efa64a1bde None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:39.221687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-9ae79407-cab4-4cac-8ebb-27efa64a1bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:39.251395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11015ac-3037-495f-bdb3-e6cdd00131b9 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:39.305902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:39.314320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 198/398] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:39.417648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84f885-bd95-4f26-b74b-abd5719d082f None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:39.418300 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84f885-bd95-4f26-b74b-abd5719d082f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:39.418614 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84f885-bd95-4f26-b74b-abd5719d082f None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:39.420526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 199/399] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:39.428484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9183b-7f4a-4301-b916-6bb8c52ecb2c None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:39.428767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9183b-7f4a-4301-b916-6bb8c52ecb2c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:39.429141 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9183b-7f4a-4301-b916-6bb8c52ecb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:39.429479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11015ac-3037-495f-bdb3-e6cdd00131b9 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:39.480573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f8e9183b-7f4a-4301-b916-6bb8c52ecb2c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:39.481696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f8e9183b-7f4a-4301-b916-6bb8c52ecb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:39.485693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7256d51-3b95-461a-ad16-c8dfb0180945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:39.486391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7256d51-3b95-461a-ad16-c8dfb0180945 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:39.486667 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7256d51-3b95-461a-ad16-c8dfb0180945 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:39.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7256d51-3b95-461a-ad16-c8dfb0180945 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:39.508074 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7256d51-3b95-461a-ad16-c8dfb0180945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-d7256d51-3b95-461a-ad16-c8dfb0180945 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:39.517946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:39.518866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:39.519711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:39.519711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:39.519711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:39.519711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:39.519711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:39.519711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:39.519711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:39.519711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:39.519711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:39.519711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:39.519711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:39.519711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:39.520938 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 201/400] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:39.529594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7256d51-3b95-461a-ad16-c8dfb0180945 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:39.574843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:39.637070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 200/401] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:39.657039 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7256d51-3b95-461a-ad16-c8dfb0180945 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:39.677359 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9183b-7f4a-4301-b916-6bb8c52ecb2c None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:39.694411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa56e5-63cb-4530-8fae-6a4f41ea56b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:39.695038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa56e5-63cb-4530-8fae-6a4f41ea56b6 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:39.695317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa56e5-63cb-4530-8fae-6a4f41ea56b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:39.695621 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa56e5-63cb-4530-8fae-6a4f41ea56b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:39.696839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa56e5-63cb-4530-8fae-6a4f41ea56b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:39.697146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa56e5-63cb-4530-8fae-6a4f41ea56b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:39.698533 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa56e5-63cb-4530-8fae-6a4f41ea56b6 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:39.708519 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 202/402] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:39.714539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa56e5-63cb-4530-8fae-6a4f41ea56b6 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:39.878844 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9183b-7f4a-4301-b916-6bb8c52ecb2c None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:39.885281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:39.903716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa56e5-63cb-4530-8fae-6a4f41ea56b6 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:39.930557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:39.949172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45619fb4-41ff-4f34-b20c-81b81725bc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:39.949931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45619fb4-41ff-4f34-b20c-81b81725bc2b None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:39.950198 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45619fb4-41ff-4f34-b20c-81b81725bc2b None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:39.950460 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45619fb4-41ff-4f34-b20c-81b81725bc2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:39.957208 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3ab4b-19ac-4df4-ae56-4278ebd51a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:39.957913 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3ab4b-19ac-4df4-ae56-4278ebd51a7c None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:39.958249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3ab4b-19ac-4df4-ae56-4278ebd51a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:39.958785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3ab4b-19ac-4df4-ae56-4278ebd51a7c None admin] PATH_INFO: `/v3/groups/d08827f6950a4a2999950a5d803b58ab` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:39.960287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3ab4b-19ac-4df4-ae56-4278ebd51a7c None admin] RBAC: Authorizing `identity:get_group(group_id=d08827f6950a4a2999950a5d803b58ab)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:39.962000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d3ab4b-19ac-4df4-ae56-4278ebd51a7c None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:39.962244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45619fb4-41ff-4f34-b20c-81b81725bc2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-45619fb4-41ff-4f34-b20c-81b81725bc2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:39.967559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:39.968454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:39.969265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:39.969265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:39.969265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:39.969265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:39.969265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:39.969265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:39.969265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:39.969265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:39.969265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:39.969265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:39.969265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:54:39.969265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:39.969876 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 203/403] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Sep 8 15:54:39 2022] GET /identity/v3/groups/d08827f6950a4a2999950a5d803b58ab => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:39.969945 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 201/404] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:54:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:54:39.970009 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3ab4b-19ac-4df4-ae56-4278ebd51a7c None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:39.976364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45619fb4-41ff-4f34-b20c-81b81725bc2b None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:40.083408 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3ab4b-19ac-4df4-ae56-4278ebd51a7c None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:40.089121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45619fb4-41ff-4f34-b20c-81b81725bc2b None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:40.121351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59269c9f-6c49-4409-9128-e3070a562064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:40.121990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59269c9f-6c49-4409-9128-e3070a562064 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:40.122262 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59269c9f-6c49-4409-9128-e3070a562064 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:40.122531 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59269c9f-6c49-4409-9128-e3070a562064 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:40.131188 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32463d-4993-4cfc-bfe3-51e777caeafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:40.131919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32463d-4993-4cfc-bfe3-51e777caeafa None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:40.132192 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32463d-4993-4cfc-bfe3-51e777caeafa None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:40.132458 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32463d-4993-4cfc-bfe3-51e777caeafa None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:40.133134 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c32463d-4993-4cfc-bfe3-51e777caeafa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:40.133428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c32463d-4993-4cfc-bfe3-51e777caeafa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:40.134811 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c32463d-4993-4cfc-bfe3-51e777caeafa None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:40.138893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59269c9f-6c49-4409-9128-e3070a562064 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:40.143572 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 202/405] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:40.144384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32463d-4993-4cfc-bfe3-51e777caeafa None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-59269c9f-6c49-4409-9128-e3070a562064 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:40.149718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:40.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:40.153385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:40.153385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:40.153385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:40.153385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:40.153385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:40.153385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:40.153385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:40.153385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:40.153385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:40.153385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:40.153385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:40.153385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:40.153385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 204/406] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:54:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:40.154652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59269c9f-6c49-4409-9128-e3070a562064 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:40.191795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:40.213944 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:40.275711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32463d-4993-4cfc-bfe3-51e777caeafa None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:40.296649 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59269c9f-6c49-4409-9128-e3070a562064 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:40.322037 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c078ce0-a11d-4ec8-8174-0552be919ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:40.323183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c078ce0-a11d-4ec8-8174-0552be919ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:40.323661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c078ce0-a11d-4ec8-8174-0552be919ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:40.324315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c078ce0-a11d-4ec8-8174-0552be919ba4 None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:40.326250 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c078ce0-a11d-4ec8-8174-0552be919ba4 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:40.328494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c078ce0-a11d-4ec8-8174-0552be919ba4 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:40.330551 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 203/407] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:40 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:40.331577 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c078ce0-a11d-4ec8-8174-0552be919ba4 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:40.339182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc284438-677e-452d-b4ed-17514a739061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:40.339980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc284438-677e-452d-b4ed-17514a739061 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:40.340376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc284438-677e-452d-b4ed-17514a739061 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:40.340736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc284438-677e-452d-b4ed-17514a739061 None admin] PATH_INFO: `/v3/groups/66861b522a6b4058bce322d39977662b` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:40.342341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc284438-677e-452d-b4ed-17514a739061 None admin] RBAC: Authorizing `identity:get_group(group_id=66861b522a6b4058bce322d39977662b)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:40.343553 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc284438-677e-452d-b4ed-17514a739061 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:40.349960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 205/408] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Sep 8 15:54:40 2022] GET /identity/v3/groups/66861b522a6b4058bce322d39977662b => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:40.350889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc284438-677e-452d-b4ed-17514a739061 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:40.442370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:40.458669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc284438-677e-452d-b4ed-17514a739061 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:40.467489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c078ce0-a11d-4ec8-8174-0552be919ba4 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:40.492660 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:40.495683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efa298-28f1-4f59-8bd8-c53d35c6b822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:40.496471 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efa298-28f1-4f59-8bd8-c53d35c6b822 None admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:40.497143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efa298-28f1-4f59-8bd8-c53d35c6b822 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:40.497143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02efa298-28f1-4f59-8bd8-c53d35c6b822 None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd/groups/d08827f6950a4a2999950a5d803b58ab/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:40.499341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02efa298-28f1-4f59-8bd8-c53d35c6b822 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb5f153291f4fb28a4ccf24849d2ddd, group_id=d08827f6950a4a2999950a5d803b58ab, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:40.501629 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02efa298-28f1-4f59-8bd8-c53d35c6b822 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:40.513741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e66e61-3157-451a-9de1-6871ca3d389e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:40.514724 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e66e61-3157-451a-9de1-6871ca3d389e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:40.515242 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e66e61-3157-451a-9de1-6871ca3d389e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:40.515739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e66e61-3157-451a-9de1-6871ca3d389e None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:40.516690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e66e61-3157-451a-9de1-6871ca3d389e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:40.517240 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e66e61-3157-451a-9de1-6871ca3d389e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:40.518895 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e66e61-3157-451a-9de1-6871ca3d389e None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:40.523526 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 206/409] 10.222.0.27 () {58 vars in 1442 bytes} [Thu Sep 8 15:54:40 2022] PUT /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd/groups/d08827f6950a4a2999950a5d803b58ab/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:54:40.524785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efa298-28f1-4f59-8bd8-c53d35c6b822 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:40.536360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 204/410] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:40.542499 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e66e61-3157-451a-9de1-6871ca3d389e None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:40.693270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e66e61-3157-451a-9de1-6871ca3d389e None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:40.739074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02121cd8-d855-4bb0-ad89-58407918e6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:40.740187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02121cd8-d855-4bb0-ad89-58407918e6be None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:40.740654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02efa298-28f1-4f59-8bd8-c53d35c6b822 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:40.740921 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02121cd8-d855-4bb0-ad89-58407918e6be None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:40.741429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02121cd8-d855-4bb0-ad89-58407918e6be None admin] PATH_INFO: `/v3/groups/d08827f6950a4a2999950a5d803b58ab` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:40.743008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02121cd8-d855-4bb0-ad89-58407918e6be None admin] RBAC: Authorizing `identity:get_group(group_id=d08827f6950a4a2999950a5d803b58ab)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:40.744840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02121cd8-d855-4bb0-ad89-58407918e6be None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:40.747198 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 205/411] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Sep 8 15:54:40 2022] GET /identity/v3/groups/d08827f6950a4a2999950a5d803b58ab => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:40.748761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f156b3-6e6f-41f4-8549-0fc29036b5d7 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:40.749275 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f156b3-6e6f-41f4-8549-0fc29036b5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:40.749738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f156b3-6e6f-41f4-8549-0fc29036b5d7 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:40.751312 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 206/412] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:40.752631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f156b3-6e6f-41f4-8549-0fc29036b5d7 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:40.793674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c1a0b-bd1f-4a93-b266-93e30c02deb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:40.794734 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c1a0b-bd1f-4a93-b266-93e30c02deb0 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:40.795015 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c1a0b-bd1f-4a93-b266-93e30c02deb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:40.795281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c1a0b-bd1f-4a93-b266-93e30c02deb0 None admin] PATH_INFO: `/v3/projects/e21b84c9f53a40aa98d4a318db545714` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:40.796607 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c1a0b-bd1f-4a93-b266-93e30c02deb0 None admin] RBAC: Authorizing `identity:get_project(project_id=e21b84c9f53a40aa98d4a318db545714)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:40.798236 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c1a0b-bd1f-4a93-b266-93e30c02deb0 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:40.800319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 207/413] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:40 2022] GET /identity/v3/projects/e21b84c9f53a40aa98d4a318db545714 => generated 344 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:40.801528 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ab41f-8476-4558-8fa1-1bb6545bd59c None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:40.801904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ab41f-8476-4558-8fa1-1bb6545bd59c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:40.802179 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ab41f-8476-4558-8fa1-1bb6545bd59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:40.832143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-779ab41f-8476-4558-8fa1-1bb6545bd59c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:40.832143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-779ab41f-8476-4558-8fa1-1bb6545bd59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:40.839654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:40.847493 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:40.908958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f156b3-6e6f-41f4-8549-0fc29036b5d7 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:40.927971 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 208/414] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:40.928832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ab41f-8476-4558-8fa1-1bb6545bd59c None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:40.966578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca1f9d-4e1f-4684-9998-9de264d399e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:40.967198 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca1f9d-4e1f-4684-9998-9de264d399e5 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:40.967553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca1f9d-4e1f-4684-9998-9de264d399e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:40.967966 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ca1f9d-4e1f-4684-9998-9de264d399e5 None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:40.972229 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca1f9d-4e1f-4684-9998-9de264d399e5 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:40.973865 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ca1f9d-4e1f-4684-9998-9de264d399e5 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:40.975938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 207/415] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:40 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:40.981682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca1f9d-4e1f-4684-9998-9de264d399e5 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:41.040567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ab41f-8476-4558-8fa1-1bb6545bd59c None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:41.079129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d602a90-ee99-4e92-b9a6-d5b19f561fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:41.079846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d602a90-ee99-4e92-b9a6-d5b19f561fe5 None admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:41.080125 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d602a90-ee99-4e92-b9a6-d5b19f561fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:41.080390 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d602a90-ee99-4e92-b9a6-d5b19f561fe5 None admin] PATH_INFO: `/v3/projects/e21b84c9f53a40aa98d4a318db545714/groups/66861b522a6b4058bce322d39977662b/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:41.082325 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d602a90-ee99-4e92-b9a6-d5b19f561fe5 None admin] RBAC: Authorizing `identity:create_grant(project_id=e21b84c9f53a40aa98d4a318db545714, group_id=66861b522a6b4058bce322d39977662b, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:41.083788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d602a90-ee99-4e92-b9a6-d5b19f561fe5 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:41.111169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 209/416] 10.222.0.27 () {58 vars in 1442 bytes} [Thu Sep 8 15:54:40 2022] PUT /identity/v3/projects/e21b84c9f53a40aa98d4a318db545714/groups/66861b522a6b4058bce322d39977662b/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:54:41.168635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ca1f9d-4e1f-4684-9998-9de264d399e5 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:41.189149 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:41.219386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e71014-387c-4d9f-9c41-9322df881447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:41.220069 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e71014-387c-4d9f-9c41-9322df881447 None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:41.220353 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e71014-387c-4d9f-9c41-9322df881447 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:41.220628 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e71014-387c-4d9f-9c41-9322df881447 None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd/groups/d08827f6950a4a2999950a5d803b58ab/roles/bf98c97818504bc48e3bd1b2aef9d3c3` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:41.223098 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e71014-387c-4d9f-9c41-9322df881447 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fb5f153291f4fb28a4ccf24849d2ddd, group_id=d08827f6950a4a2999950a5d803b58ab, role_id=bf98c97818504bc48e3bd1b2aef9d3c3)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:41.224663 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e71014-387c-4d9f-9c41-9322df881447 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:41.249971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 208/417] 10.222.0.27 () {58 vars in 1442 bytes} [Thu Sep 8 15:54:40 2022] PUT /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd/groups/d08827f6950a4a2999950a5d803b58ab/roles/bf98c97818504bc48e3bd1b2aef9d3c3 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 15:54:41.264719 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071fe5e-7341-42de-ab18-5bf687b5f88e None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:41.265058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071fe5e-7341-42de-ab18-5bf687b5f88e None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:41.265276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1071fe5e-7341-42de-ab18-5bf687b5f88e None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:41.266961 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 210/418] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:41.273318 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86887130-89e0-4145-98bd-e0d5ecd98a80 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:41.273609 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86887130-89e0-4145-98bd-e0d5ecd98a80 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:41.273876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86887130-89e0-4145-98bd-e0d5ecd98a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:41.294421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-86887130-89e0-4145-98bd-e0d5ecd98a80 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:41.295768 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-86887130-89e0-4145-98bd-e0d5ecd98a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:41.307938 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071fe5e-7341-42de-ab18-5bf687b5f88e None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:41.313323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:41.416836 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 209/419] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:41.457504 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1071fe5e-7341-42de-ab18-5bf687b5f88e None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:41.499339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b20b89-5f9c-48e9-a8ac-b858173bbf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:41.499942 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b20b89-5f9c-48e9-a8ac-b858173bbf31 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:41.500219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b20b89-5f9c-48e9-a8ac-b858173bbf31 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:41.500489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b20b89-5f9c-48e9-a8ac-b858173bbf31 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:41.514299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b20b89-5f9c-48e9-a8ac-b858173bbf31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-38b20b89-5f9c-48e9-a8ac-b858173bbf31 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:41.519381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:41.520287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:41.521079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:41.521079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:41.521079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:41.521079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:41.521079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:41.521079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:41.521079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:41.521079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:41.521079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:41.521079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:41.521079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:41.521079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:41.521079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 211/420] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:54:41.532876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86887130-89e0-4145-98bd-e0d5ecd98a80 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:41.649356 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86887130-89e0-4145-98bd-e0d5ecd98a80 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:41.661976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:41.697419 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33583fad-c880-4bd4-a287-8dde9e16bc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:41.698461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33583fad-c880-4bd4-a287-8dde9e16bc24 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:41.698931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33583fad-c880-4bd4-a287-8dde9e16bc24 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:41.699392 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33583fad-c880-4bd4-a287-8dde9e16bc24 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:41.700296 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33583fad-c880-4bd4-a287-8dde9e16bc24 None admin] RBAC: Adding query filter params (name=member) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:41.700800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33583fad-c880-4bd4-a287-8dde9e16bc24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:41.702382 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33583fad-c880-4bd4-a287-8dde9e16bc24 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:41.715582 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 210/421] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:41.721636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b20b89-5f9c-48e9-a8ac-b858173bbf31 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:41.733872 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33583fad-c880-4bd4-a287-8dde9e16bc24 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:41.861062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33583fad-c880-4bd4-a287-8dde9e16bc24 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:41.876294 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:41.928282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18a484-627a-49f8-8447-802de31562f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:41.928929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18a484-627a-49f8-8447-802de31562f6 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:41.929197 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18a484-627a-49f8-8447-802de31562f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:41.929461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be18a484-627a-49f8-8447-802de31562f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:41.940723 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b20b89-5f9c-48e9-a8ac-b858173bbf31 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:41.952441 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be18a484-627a-49f8-8447-802de31562f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-be18a484-627a-49f8-8447-802de31562f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:41.958496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:41.959475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:41.960281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:41.960281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:41.960281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:41.960281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:41.960281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:41.960281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:41.960281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:41.960281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:41.960281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:41.960281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:41.960281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:41.960281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:41.960281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 211/422] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:54:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:41.968457 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18a484-627a-49f8-8447-802de31562f6 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:42.002167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e53e24-50bb-48f4-b34e-57496d8409a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:42.002785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e53e24-50bb-48f4-b34e-57496d8409a2 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:42.003134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e53e24-50bb-48f4-b34e-57496d8409a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:42.003400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e53e24-50bb-48f4-b34e-57496d8409a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:42.018598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:42.083052 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e53e24-50bb-48f4-b34e-57496d8409a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:42.084831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e53e24-50bb-48f4-b34e-57496d8409a2 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:42.127410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be18a484-627a-49f8-8447-802de31562f6 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:42.158269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:42.216213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcc5c8-23fc-4722-a132-622863fb8ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:42.218315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcc5c8-23fc-4722-a132-622863fb8ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:42.219175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcc5c8-23fc-4722-a132-622863fb8ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:42.219869 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcc5c8-23fc-4722-a132-622863fb8ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:42.220095 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 212/423] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:42.220975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcc5c8-23fc-4722-a132-622863fb8ae1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:42.221483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcc5c8-23fc-4722-a132-622863fb8ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:42.223110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcc5c8-23fc-4722-a132-622863fb8ae1 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:42.224653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e53e24-50bb-48f4-b34e-57496d8409a2 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:42.244521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 212/424] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:42.252071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcc5c8-23fc-4722-a132-622863fb8ae1 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:42.399920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e53e24-50bb-48f4-b34e-57496d8409a2 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:42.441063 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:42.447322 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4544968-3444-4190-9ec4-f8b4fa4caae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:42.448239 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4544968-3444-4190-9ec4-f8b4fa4caae6 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:42.448728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4544968-3444-4190-9ec4-f8b4fa4caae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:42.449184 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4544968-3444-4190-9ec4-f8b4fa4caae6 None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:42.450530 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4544968-3444-4190-9ec4-f8b4fa4caae6 None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:42.452417 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4544968-3444-4190-9ec4-f8b4fa4caae6 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:42.454850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 213/425] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:42 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:42.460013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4544968-3444-4190-9ec4-f8b4fa4caae6 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:42.481804 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcc5c8-23fc-4722-a132-622863fb8ae1 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:42.515878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:42.575421 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4544968-3444-4190-9ec4-f8b4fa4caae6 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:42.592693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1dc9b-6d27-4ae8-965f-ff2ff50d3c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:42.593339 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1dc9b-6d27-4ae8-965f-ff2ff50d3c4b None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:42.593588 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1dc9b-6d27-4ae8-965f-ff2ff50d3c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:42.593858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1dc9b-6d27-4ae8-965f-ff2ff50d3c4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:42.627294 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f1e0a-4404-4005-abcb-01839640d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:42.628453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f1e0a-4404-4005-abcb-01839640d338 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:42.628946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f1e0a-4404-4005-abcb-01839640d338 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:42.629440 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8f1e0a-4404-4005-abcb-01839640d338 None admin] PATH_INFO: `/v3/groups/d08827f6950a4a2999950a5d803b58ab` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:42.634229 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f1e0a-4404-4005-abcb-01839640d338 None admin] RBAC: Authorizing `identity:get_group(group_id=d08827f6950a4a2999950a5d803b58ab)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:42.635928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8f1e0a-4404-4005-abcb-01839640d338 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:42.637747 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 214/426] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Sep 8 15:54:42 2022] GET /identity/v3/groups/d08827f6950a4a2999950a5d803b58ab => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:42.643541 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f1e0a-4404-4005-abcb-01839640d338 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:42.687979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1dc9b-6d27-4ae8-965f-ff2ff50d3c4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:42.687979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1dc9b-6d27-4ae8-965f-ff2ff50d3c4b None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:42.787338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8f1e0a-4404-4005-abcb-01839640d338 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:42.799710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:42.820374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:42.836058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e9346-8d24-4c04-94f2-57dadd922351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:42.836058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e9346-8d24-4c04-94f2-57dadd922351 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:42.836058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e9346-8d24-4c04-94f2-57dadd922351 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:42.836058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e9346-8d24-4c04-94f2-57dadd922351 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:42.836491 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e9346-8d24-4c04-94f2-57dadd922351 None admin] RBAC: Adding query filter params (group.id=d08827f6950a4a2999950a5d803b58ab, role.id=abbd88ef58a544db8ee4261548778980, scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:42.836491 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e9346-8d24-4c04-94f2-57dadd922351 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:42.838204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e9346-8d24-4c04-94f2-57dadd922351 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:42.846386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 213/427] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:42.856918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1dc9b-6d27-4ae8-965f-ff2ff50d3c4b None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:42.865361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 215/428] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Sep 8 15:54:42 2022] GET /identity/v3/role_assignments?group.id=d08827f6950a4a2999950a5d803b58ab&scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e&role.id=abbd88ef58a544db8ee4261548778980 => generated 611 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:42.965195 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1dc9b-6d27-4ae8-965f-ff2ff50d3c4b None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:43.007188 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f71db1-148a-46e0-8d6d-fa086a0968c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:43.008314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f71db1-148a-46e0-8d6d-fa086a0968c6 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:43.008792 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f71db1-148a-46e0-8d6d-fa086a0968c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:43.009264 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f71db1-148a-46e0-8d6d-fa086a0968c6 None admin] PATH_INFO: `/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:43.010972 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f71db1-148a-46e0-8d6d-fa086a0968c6 None admin] RBAC: Authorizing `identity:get_project(project_id=ae75fe116ff145cdb11b0b2127b2ff7e)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:43.012770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f71db1-148a-46e0-8d6d-fa086a0968c6 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:43.014775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 214/429] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:42 2022] GET /identity/v3/projects/ae75fe116ff145cdb11b0b2127b2ff7e => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:43.024636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e9346-8d24-4c04-94f2-57dadd922351 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:43.159584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5bd17-1025-411d-9f70-1db4c27966fe None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:43.159805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e9346-8d24-4c04-94f2-57dadd922351 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:43.160366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5bd17-1025-411d-9f70-1db4c27966fe None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:43.160842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5bd17-1025-411d-9f70-1db4c27966fe None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:43.162707 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 215/430] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:43.169565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2ec2a-a75c-472a-8b5e-85bcb71d8da5 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:43.170166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2ec2a-a75c-472a-8b5e-85bcb71d8da5 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:43.170646 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a2ec2a-a75c-472a-8b5e-85bcb71d8da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:43.204306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f9a2ec2a-a75c-472a-8b5e-85bcb71d8da5 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:43.206067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f9a2ec2a-a75c-472a-8b5e-85bcb71d8da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:43.220806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefeac7c-8b78-4cf2-8267-5319a4efc668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:43.221382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefeac7c-8b78-4cf2-8267-5319a4efc668 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:43.221722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefeac7c-8b78-4cf2-8267-5319a4efc668 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:43.221997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefeac7c-8b78-4cf2-8267-5319a4efc668 None admin] PATH_INFO: `/v3/groups/d08827f6950a4a2999950a5d803b58ab` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:43.223762 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefeac7c-8b78-4cf2-8267-5319a4efc668 None admin] RBAC: Authorizing `identity:get_group(group_id=d08827f6950a4a2999950a5d803b58ab)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:43.225203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefeac7c-8b78-4cf2-8267-5319a4efc668 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:43.227044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 216/431] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Sep 8 15:54:43 2022] GET /identity/v3/groups/d08827f6950a4a2999950a5d803b58ab => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:43.233992 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefeac7c-8b78-4cf2-8267-5319a4efc668 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:43.278220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:43.284892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:43.304691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 216/432] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:43.370957 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefeac7c-8b78-4cf2-8267-5319a4efc668 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:43.407796 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a5bde-968a-4cf0-ae58-e45493d366e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:43.408469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a5bde-968a-4cf0-ae58-e45493d366e6 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:43.408748 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a5bde-968a-4cf0-ae58-e45493d366e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:43.409021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a5bde-968a-4cf0-ae58-e45493d366e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:43.409833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a5bde-968a-4cf0-ae58-e45493d366e6 None admin] RBAC: Adding query filter params (group.id=d08827f6950a4a2999950a5d803b58ab, role.id=bf98c97818504bc48e3bd1b2aef9d3c3, scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:43.410136 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a5bde-968a-4cf0-ae58-e45493d366e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:43.411643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a5bde-968a-4cf0-ae58-e45493d366e6 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:43.418929 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 217/433] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Sep 8 15:54:43 2022] GET /identity/v3/role_assignments?group.id=d08827f6950a4a2999950a5d803b58ab&scope.project.id=ae75fe116ff145cdb11b0b2127b2ff7e&role.id=bf98c97818504bc48e3bd1b2aef9d3c3 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:43.613153 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2ec2a-a75c-472a-8b5e-85bcb71d8da5 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:43.729934 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a2ec2a-a75c-472a-8b5e-85bcb71d8da5 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:43.773446 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a59db-e35e-4745-9d9f-40671fbeb756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:43.774122 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a59db-e35e-4745-9d9f-40671fbeb756 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:43.774399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a59db-e35e-4745-9d9f-40671fbeb756 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:43.774667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a59db-e35e-4745-9d9f-40671fbeb756 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:43.789973 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a59db-e35e-4745-9d9f-40671fbeb756 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-6b0a59db-e35e-4745-9d9f-40671fbeb756 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:43.795108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:43.796105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:43.796909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:43.796909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:43.796909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:43.796909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:43.796909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:43.796909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:43.796909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:43.796909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:43.796909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:43.796909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:43.796909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 15:54:43.796909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:43.796909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 217/434] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:43.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a5bde-968a-4cf0-ae58-e45493d366e6 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:43.874703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9739a-68ae-431b-8e10-32e5ffd33dfc None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:43.874873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9739a-68ae-431b-8e10-32e5ffd33dfc None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:43.875136 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9739a-68ae-431b-8e10-32e5ffd33dfc None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:43.877858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 218/435] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:54:43.883173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1ed86-f9a9-471e-8bd1-8f04541e2448 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:43.884676 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1ed86-f9a9-471e-8bd1-8f04541e2448 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:43.884676 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1ed86-f9a9-471e-8bd1-8f04541e2448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:43.908739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-62f1ed86-f9a9-471e-8bd1-8f04541e2448 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:43.910394 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-62f1ed86-f9a9-471e-8bd1-8f04541e2448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:43.911348 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:43.918587 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a5bde-968a-4cf0-ae58-e45493d366e6 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:43.953923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5435f8e6-651e-4b83-b951-c7b0bb0ec6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:43.954545 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435f8e6-651e-4b83-b951-c7b0bb0ec6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:43.954856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435f8e6-651e-4b83-b951-c7b0bb0ec6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:43.955086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5435f8e6-651e-4b83-b951-c7b0bb0ec6c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:43.955836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435f8e6-651e-4b83-b951-c7b0bb0ec6c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:43.956171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435f8e6-651e-4b83-b951-c7b0bb0ec6c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:43.957558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5435f8e6-651e-4b83-b951-c7b0bb0ec6c3 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:43.966953 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 218/436] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:54:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:43.968279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a73097-4d82-4e21-869d-3ccce129dfd1 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:43.968562 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a73097-4d82-4e21-869d-3ccce129dfd1 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:43.968833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a73097-4d82-4e21-869d-3ccce129dfd1 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:43.970059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 219/437] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:43.971902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:43.975027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a73097-4d82-4e21-869d-3ccce129dfd1 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:43.998959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 219/438] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:44.000504 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9aa5fc-1a23-4259-90ee-80ebff5e0044 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:44.000868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9aa5fc-1a23-4259-90ee-80ebff5e0044 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:44.001206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9aa5fc-1a23-4259-90ee-80ebff5e0044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:44.021391 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-8e9aa5fc-1a23-4259-90ee-80ebff5e0044 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:44.023269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8e9aa5fc-1a23-4259-90ee-80ebff5e0044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:44.083816 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a73097-4d82-4e21-869d-3ccce129dfd1 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:44.099732 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 220/439] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:44.117795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1fa4b-68a3-4707-b970-41c3e504b2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:44.118406 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1fa4b-68a3-4707-b970-41c3e504b2a1 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:44.118678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1fa4b-68a3-4707-b970-41c3e504b2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:44.118942 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1fa4b-68a3-4707-b970-41c3e504b2a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:44.146679 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:44.193364 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1fa4b-68a3-4707-b970-41c3e504b2a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:44.194838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1fa4b-68a3-4707-b970-41c3e504b2a1 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:44.273694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9aa5fc-1a23-4259-90ee-80ebff5e0044 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:44.297317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 220/440] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:44.303658 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1fa4b-68a3-4707-b970-41c3e504b2a1 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:44.364015 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:44.379980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9aa5fc-1a23-4259-90ee-80ebff5e0044 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:44.389079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:44.404373 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1fa4b-68a3-4707-b970-41c3e504b2a1 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:44.413754 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b7564-6258-429f-be95-9871e32ec827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:44.414374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b7564-6258-429f-be95-9871e32ec827 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:44.414655 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b7564-6258-429f-be95-9871e32ec827 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:44.414922 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b7564-6258-429f-be95-9871e32ec827 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:44.425402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b7564-6258-429f-be95-9871e32ec827 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-df6b7564-6258-429f-be95-9871e32ec827 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:44.430257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:44.431144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:44.432308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:44.432308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:44.432308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:44.432308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:44.432308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:44.432308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:44.432308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:44.432308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:44.432308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:44.432308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:44.432308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:54:44.432308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:44.432308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 221/441] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:54:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:54:44.434113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b7564-6258-429f-be95-9871e32ec827 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:44.437462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec8058-516a-471f-adcf-e081f8005870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:44.438101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec8058-516a-471f-adcf-e081f8005870 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:44.438375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec8058-516a-471f-adcf-e081f8005870 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:44.438643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec8058-516a-471f-adcf-e081f8005870 None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:44.439799 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec8058-516a-471f-adcf-e081f8005870 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:44.441302 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec8058-516a-471f-adcf-e081f8005870 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:44.445549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 221/442] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:44 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:44.445549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec8058-516a-471f-adcf-e081f8005870 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:44.535942 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b7564-6258-429f-be95-9871e32ec827 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:44.544609 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec8058-516a-471f-adcf-e081f8005870 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:44.570221 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:44.570715 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e698f-c61d-434b-92aa-b1b2352ba9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:44.571367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e698f-c61d-434b-92aa-b1b2352ba9ce None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:44.571742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e698f-c61d-434b-92aa-b1b2352ba9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:44.572018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e698f-c61d-434b-92aa-b1b2352ba9ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:44.578742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442fb6c-36be-471b-a10a-28ee76acc8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:44.579344 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442fb6c-36be-471b-a10a-28ee76acc8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:44.579650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442fb6c-36be-471b-a10a-28ee76acc8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:44.579916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442fb6c-36be-471b-a10a-28ee76acc8c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:44.580572 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442fb6c-36be-471b-a10a-28ee76acc8c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:44.580867 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442fb6c-36be-471b-a10a-28ee76acc8c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:44.582246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e698f-c61d-434b-92aa-b1b2352ba9ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:44.582347 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442fb6c-36be-471b-a10a-28ee76acc8c3 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-3b3e698f-c61d-434b-92aa-b1b2352ba9ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:44.587035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:44.588315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:44.589096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:44.589096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:44.589096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:44.589096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:44.589096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:44.589096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:44.589096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:44.589096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:44.589096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:44.589096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:44.589096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:44.589096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:44.589096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 222/443] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:54:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:44.589739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e698f-c61d-434b-92aa-b1b2352ba9ce None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:44.593558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 222/444] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:44.594385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442fb6c-36be-471b-a10a-28ee76acc8c3 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:44.618962 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:44.697313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442fb6c-36be-471b-a10a-28ee76acc8c3 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:44.702409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e698f-c61d-434b-92aa-b1b2352ba9ce None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:44.736353 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7de94c-f6b1-4119-92d5-5e19208d8656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:44.736972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7de94c-f6b1-4119-92d5-5e19208d8656 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:44.737248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7de94c-f6b1-4119-92d5-5e19208d8656 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:44.737516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7de94c-f6b1-4119-92d5-5e19208d8656 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:44.737969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811acde-b9e0-433d-b48e-13337031a034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:44.738192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7de94c-f6b1-4119-92d5-5e19208d8656 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:44.738495 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7de94c-f6b1-4119-92d5-5e19208d8656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:44.738589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811acde-b9e0-433d-b48e-13337031a034 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:44.738861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811acde-b9e0-433d-b48e-13337031a034 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:44.739127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811acde-b9e0-433d-b48e-13337031a034 None admin] PATH_INFO: `/v3/groups/d08827f6950a4a2999950a5d803b58ab` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:44.739922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7de94c-f6b1-4119-92d5-5e19208d8656 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:44.740441 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811acde-b9e0-433d-b48e-13337031a034 None admin] RBAC: Authorizing `identity:get_group(group_id=d08827f6950a4a2999950a5d803b58ab)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:44.741904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e811acde-b9e0-433d-b48e-13337031a034 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:44.743565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 223/445] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Sep 8 15:54:44 2022] GET /identity/v3/groups/d08827f6950a4a2999950a5d803b58ab => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:44.744323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811acde-b9e0-433d-b48e-13337031a034 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:44.753002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 223/446] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:44.753002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7de94c-f6b1-4119-92d5-5e19208d8656 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:44.804941 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:44.838396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:44.844741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e811acde-b9e0-433d-b48e-13337031a034 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:44.855648 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7de94c-f6b1-4119-92d5-5e19208d8656 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:44.878458 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322525cb-f583-44c5-a9e6-ee4d7170618d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:44.879262 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322525cb-f583-44c5-a9e6-ee4d7170618d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:44.879570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322525cb-f583-44c5-a9e6-ee4d7170618d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:44.879843 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322525cb-f583-44c5-a9e6-ee4d7170618d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:44.887771 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec20566-7cdc-497e-b06a-f88d9609199a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:44.888380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec20566-7cdc-497e-b06a-f88d9609199a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:44.888650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec20566-7cdc-497e-b06a-f88d9609199a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:44.888918 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec20566-7cdc-497e-b06a-f88d9609199a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:44.889640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec20566-7cdc-497e-b06a-f88d9609199a None admin] RBAC: Adding query filter params (group.id=d08827f6950a4a2999950a5d803b58ab, role.id=abbd88ef58a544db8ee4261548778980, scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:44.889989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec20566-7cdc-497e-b06a-f88d9609199a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:44.891406 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec20566-7cdc-497e-b06a-f88d9609199a None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:44.897177 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 224/447] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Sep 8 15:54:44 2022] GET /identity/v3/role_assignments?group.id=d08827f6950a4a2999950a5d803b58ab&scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd&role.id=abbd88ef58a544db8ee4261548778980 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:44.897949 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec20566-7cdc-497e-b06a-f88d9609199a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:44.960947 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322525cb-f583-44c5-a9e6-ee4d7170618d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:44.963517 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322525cb-f583-44c5-a9e6-ee4d7170618d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:45.014785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec20566-7cdc-497e-b06a-f88d9609199a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:45.030512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:45.055089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31bb32-17aa-4855-8ea2-52010873b3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:45.055748 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31bb32-17aa-4855-8ea2-52010873b3cb None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:45.056021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31bb32-17aa-4855-8ea2-52010873b3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:45.056403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec31bb32-17aa-4855-8ea2-52010873b3cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:45.074036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:45.082226 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 224/448] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:45.088488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322525cb-f583-44c5-a9e6-ee4d7170618d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:45.133635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31bb32-17aa-4855-8ea2-52010873b3cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:45.135310 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec31bb32-17aa-4855-8ea2-52010873b3cb None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:45.208198 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322525cb-f583-44c5-a9e6-ee4d7170618d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:45.248637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65314595-1406-4c49-ac87-921e40252282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:45.249195 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65314595-1406-4c49-ac87-921e40252282 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:45.249465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65314595-1406-4c49-ac87-921e40252282 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:45.249730 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65314595-1406-4c49-ac87-921e40252282 None admin] PATH_INFO: `/v3/projects/e21b84c9f53a40aa98d4a318db545714` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:45.250824 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65314595-1406-4c49-ac87-921e40252282 None admin] RBAC: Authorizing `identity:get_project(project_id=e21b84c9f53a40aa98d4a318db545714)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:45.252429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65314595-1406-4c49-ac87-921e40252282 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:45.254413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 225/449] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:45 2022] GET /identity/v3/projects/e21b84c9f53a40aa98d4a318db545714 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:45.257286 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 225/450] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:45.260227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65314595-1406-4c49-ac87-921e40252282 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:45.262463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31bb32-17aa-4855-8ea2-52010873b3cb None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:45.277896 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:45.327222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:45.364715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec31bb32-17aa-4855-8ea2-52010873b3cb None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:45.367233 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65314595-1406-4c49-ac87-921e40252282 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:45.397098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf516d3-f624-45de-8c75-b4d94a896564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:45.397725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf516d3-f624-45de-8c75-b4d94a896564 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:45.398004 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf516d3-f624-45de-8c75-b4d94a896564 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:45.398269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf516d3-f624-45de-8c75-b4d94a896564 None admin] PATH_INFO: `/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:45.399496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf516d3-f624-45de-8c75-b4d94a896564 None admin] RBAC: Authorizing `identity:get_project(project_id=3fb5f153291f4fb28a4ccf24849d2ddd)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:45.400227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c511a-a46e-466a-971b-ed872859b684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:45.400856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c511a-a46e-466a-971b-ed872859b684 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:45.400943 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf516d3-f624-45de-8c75-b4d94a896564 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:45.401134 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c511a-a46e-466a-971b-ed872859b684 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:45.401406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c511a-a46e-466a-971b-ed872859b684 None admin] PATH_INFO: `/v3/groups/66861b522a6b4058bce322d39977662b` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:45.402519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c511a-a46e-466a-971b-ed872859b684 None admin] RBAC: Authorizing `identity:get_group(group_id=66861b522a6b4058bce322d39977662b)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:45.402657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 226/451] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:54:45 2022] GET /identity/v3/projects/3fb5f153291f4fb28a4ccf24849d2ddd => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:45.404000 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396c511a-a46e-466a-971b-ed872859b684 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:45.405556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 226/452] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Sep 8 15:54:45 2022] GET /identity/v3/groups/66861b522a6b4058bce322d39977662b => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:45.407992 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf516d3-f624-45de-8c75-b4d94a896564 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:45.411117 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c511a-a46e-466a-971b-ed872859b684 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:45.489761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:45.508086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf516d3-f624-45de-8c75-b4d94a896564 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:45.511257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c511a-a46e-466a-971b-ed872859b684 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:45.540536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f02ba-ce5d-4fb9-9685-d54772e21242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:45.540783 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:45.541214 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f02ba-ce5d-4fb9-9685-d54772e21242 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:45.541499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f02ba-ce5d-4fb9-9685-d54772e21242 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:45.541857 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f02ba-ce5d-4fb9-9685-d54772e21242 None admin] PATH_INFO: `/v3/groups/d08827f6950a4a2999950a5d803b58ab` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:45.543073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f02ba-ce5d-4fb9-9685-d54772e21242 None admin] RBAC: Authorizing `identity:get_group(group_id=d08827f6950a4a2999950a5d803b58ab)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:45.544677 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f02ba-ce5d-4fb9-9685-d54772e21242 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:45.546260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 227/453] 10.222.0.27 () {56 vars in 1180 bytes} [Thu Sep 8 15:54:45 2022] GET /identity/v3/groups/d08827f6950a4a2999950a5d803b58ab => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:45.546378 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1b19a7-38e9-4577-9b2c-04292b1e9cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:45.546970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b19a7-38e9-4577-9b2c-04292b1e9cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:45.547263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b19a7-38e9-4577-9b2c-04292b1e9cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:45.547560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1b19a7-38e9-4577-9b2c-04292b1e9cb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:45.548285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b19a7-38e9-4577-9b2c-04292b1e9cb8 None admin] RBAC: Adding query filter params (group.id=66861b522a6b4058bce322d39977662b, role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=e21b84c9f53a40aa98d4a318db545714) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:45.548590 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b19a7-38e9-4577-9b2c-04292b1e9cb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:45.550010 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1b19a7-38e9-4577-9b2c-04292b1e9cb8 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:45.552017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f02ba-ce5d-4fb9-9685-d54772e21242 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:45.571488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 227/454] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Sep 8 15:54:45 2022] GET /identity/v3/role_assignments?group.id=66861b522a6b4058bce322d39977662b&scope.project.id=e21b84c9f53a40aa98d4a318db545714&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:54:45.661353 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f02ba-ce5d-4fb9-9685-d54772e21242 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:45.696913 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d2ccf-4e93-4f94-a8b2-83077af58d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:45.697540 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d2ccf-4e93-4f94-a8b2-83077af58d29 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:45.697817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d2ccf-4e93-4f94-a8b2-83077af58d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:45.698086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d2ccf-4e93-4f94-a8b2-83077af58d29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:45.698820 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d2ccf-4e93-4f94-a8b2-83077af58d29 None admin] RBAC: Adding query filter params (group.id=d08827f6950a4a2999950a5d803b58ab, role.id=bf98c97818504bc48e3bd1b2aef9d3c3, scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:45.699130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d2ccf-4e93-4f94-a8b2-83077af58d29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:45.700644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d2ccf-4e93-4f94-a8b2-83077af58d29 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:45.707733 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 228/455] 10.222.0.27 () {56 vars in 1376 bytes} [Thu Sep 8 15:54:45 2022] GET /identity/v3/role_assignments?group.id=d08827f6950a4a2999950a5d803b58ab&scope.project.id=3fb5f153291f4fb28a4ccf24849d2ddd&role.id=bf98c97818504bc48e3bd1b2aef9d3c3 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:48.343365 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482803e-6ee0-4c77-b13d-d65b30365049 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:48.344451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482803e-6ee0-4c77-b13d-d65b30365049 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:48.344964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482803e-6ee0-4c77-b13d-d65b30365049 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:48.346797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 228/456] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:48.352581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb9d6f-f0dc-47ac-b424-7d52b901d166 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:48.353365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb9d6f-f0dc-47ac-b424-7d52b901d166 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:48.353838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbb9d6f-f0dc-47ac-b424-7d52b901d166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:48.380401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-abbb9d6f-f0dc-47ac-b424-7d52b901d166 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:48.381931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-abbb9d6f-f0dc-47ac-b424-7d52b901d166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:48.395482 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:48.481382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 229/457] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:48.636443 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0fb6c-426b-49f7-9152-38fb5386757d None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:48.637075 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0fb6c-426b-49f7-9152-38fb5386757d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:48.637575 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b0fb6c-426b-49f7-9152-38fb5386757d None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:48.639435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 229/458] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:48.644326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e206b8-f9aa-4fc1-bf79-fd903c3c8df3 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:48.644632 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e206b8-f9aa-4fc1-bf79-fd903c3c8df3 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:48.644878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e206b8-f9aa-4fc1-bf79-fd903c3c8df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:48.668009 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-d1e206b8-f9aa-4fc1-bf79-fd903c3c8df3 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:48.669189 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d1e206b8-f9aa-4fc1-bf79-fd903c3c8df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:48.722611 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160de9f6-c8a7-4718-8875-83aa709de428 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:48.723250 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160de9f6-c8a7-4718-8875-83aa709de428 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:48.723619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160de9f6-c8a7-4718-8875-83aa709de428 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:48.726074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 230/459] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:48.729963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9ac18-1760-478e-a7ee-a8bb7b9d9418 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:48.730309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9ac18-1760-478e-a7ee-a8bb7b9d9418 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:48.730652 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e9ac18-1760-478e-a7ee-a8bb7b9d9418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:48.760364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-d1e9ac18-1760-478e-a7ee-a8bb7b9d9418 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:48.761995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d1e9ac18-1760-478e-a7ee-a8bb7b9d9418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:48.780119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 230/460] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:48.782063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e206b8-f9aa-4fc1-bf79-fd903c3c8df3 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:48.893125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 231/461] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:48.916617 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e206b8-f9aa-4fc1-bf79-fd903c3c8df3 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:48.923343 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:48.975472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414b368-81fd-4112-9dc5-65728b2a71c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:48.976411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414b368-81fd-4112-9dc5-65728b2a71c5 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:48.976883 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414b368-81fd-4112-9dc5-65728b2a71c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:48.977318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414b368-81fd-4112-9dc5-65728b2a71c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:48.994157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414b368-81fd-4112-9dc5-65728b2a71c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:48.995869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414b368-81fd-4112-9dc5-65728b2a71c5 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-1414b368-81fd-4112-9dc5-65728b2a71c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:54:49.010885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:49.011829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:54:49.012703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:54:49.012703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:49.012703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:49.014397 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 231/462] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:49.020801 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9ac18-1760-478e-a7ee-a8bb7b9d9418 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:49.142151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414b368-81fd-4112-9dc5-65728b2a71c5 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:49.152452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e9ac18-1760-478e-a7ee-a8bb7b9d9418 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:49.184942 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:49.200423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d64a7-211e-4832-ab3b-884ef9477398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:49.201365 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d64a7-211e-4832-ab3b-884ef9477398 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:49.202031 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d64a7-211e-4832-ab3b-884ef9477398 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:49.203426 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d64a7-211e-4832-ab3b-884ef9477398 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:49.204323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d64a7-211e-4832-ab3b-884ef9477398 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:49.204818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d64a7-211e-4832-ab3b-884ef9477398 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:49.206476 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d64a7-211e-4832-ab3b-884ef9477398 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:49.232697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 232/463] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:49.237327 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d64a7-211e-4832-ab3b-884ef9477398 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:49.282617 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414b368-81fd-4112-9dc5-65728b2a71c5 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:49.311698 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:49.323179 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0dedb9-d24f-4944-9f0a-90e118d5df0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:49.324091 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0dedb9-d24f-4944-9f0a-90e118d5df0d None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:49.324615 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0dedb9-d24f-4944-9f0a-90e118d5df0d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:49.325029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0dedb9-d24f-4944-9f0a-90e118d5df0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:49.338100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0dedb9-d24f-4944-9f0a-90e118d5df0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:49.340229 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0dedb9-d24f-4944-9f0a-90e118d5df0d None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-be0dedb9-d24f-4944-9f0a-90e118d5df0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:54:49.358543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:49.360119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:54:49.366394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:54:49.366394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:49.366394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:49.366394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 232/464] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:54:49.366394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0dedb9-d24f-4944-9f0a-90e118d5df0d None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:49.368384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d64a7-211e-4832-ab3b-884ef9477398 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:49.407882 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73dce4-6e5d-4fe7-abc7-ace640d5960f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:49.408762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73dce4-6e5d-4fe7-abc7-ace640d5960f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:49.409236 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73dce4-6e5d-4fe7-abc7-ace640d5960f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:49.409643 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73dce4-6e5d-4fe7-abc7-ace640d5960f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:49.420285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73dce4-6e5d-4fe7-abc7-ace640d5960f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:49.421929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73dce4-6e5d-4fe7-abc7-ace640d5960f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-2e73dce4-6e5d-4fe7-abc7-ace640d5960f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:54:49.438445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:49.439389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:54:49.440749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:54:49.440749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:49.440749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:49.442392 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 233/465] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:49.443494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73dce4-6e5d-4fe7-abc7-ace640d5960f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:49.486112 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0dedb9-d24f-4944-9f0a-90e118d5df0d None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:49.498063 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:49.522660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28a274-1a7e-4010-82ca-b742900fa504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:49.523274 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28a274-1a7e-4010-82ca-b742900fa504 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:49.523586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28a274-1a7e-4010-82ca-b742900fa504 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:49.523863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28a274-1a7e-4010-82ca-b742900fa504 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:49.524520 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28a274-1a7e-4010-82ca-b742900fa504 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:49.525957 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28a274-1a7e-4010-82ca-b742900fa504 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:49.552144 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 233/466] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:54:49 2022] POST /identity/v3/users => generated 246 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:54:49.553506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28a274-1a7e-4010-82ca-b742900fa504 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:49.571663 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73dce4-6e5d-4fe7-abc7-ace640d5960f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:49.603248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:49.615717 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5715b-ee56-4713-988a-6e96e631638c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:49.617255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5715b-ee56-4713-988a-6e96e631638c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:49.617950 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5715b-ee56-4713-988a-6e96e631638c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:49.618399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5715b-ee56-4713-988a-6e96e631638c None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:49.619223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5715b-ee56-4713-988a-6e96e631638c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:49.619806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5715b-ee56-4713-988a-6e96e631638c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:49.621505 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5715b-ee56-4713-988a-6e96e631638c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:49.635826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 234/467] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:49.637452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c9727-9bd2-4198-bd48-a2519f73781f None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:49.637940 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c9727-9bd2-4198-bd48-a2519f73781f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:49.638361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6c9727-9bd2-4198-bd48-a2519f73781f None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:49.640176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 235/468] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:49.642683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c9727-9bd2-4198-bd48-a2519f73781f None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:49.705699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28a274-1a7e-4010-82ca-b742900fa504 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:49.757705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0680b483-b256-4b07-b09d-96e6c44dbbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:49.758366 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680b483-b256-4b07-b09d-96e6c44dbbb7 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:49.758644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680b483-b256-4b07-b09d-96e6c44dbbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:49.758914 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0680b483-b256-4b07-b09d-96e6c44dbbb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:49.759625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680b483-b256-4b07-b09d-96e6c44dbbb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:49.759928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680b483-b256-4b07-b09d-96e6c44dbbb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:49.761355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0680b483-b256-4b07-b09d-96e6c44dbbb7 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:49.768247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:49.774120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6c9727-9bd2-4198-bd48-a2519f73781f None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:49.775200 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 234/469] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:49.776749 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b8e04-baa4-4ae1-b543-7b8ce6c9bc27 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:49.777038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b8e04-baa4-4ae1-b543-7b8ce6c9bc27 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:49.777308 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b8e04-baa4-4ae1-b543-7b8ce6c9bc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:49.798151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-f27b8e04-baa4-4ae1-b543-7b8ce6c9bc27 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:49.799351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f27b8e04-baa4-4ae1-b543-7b8ce6c9bc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:49.812076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef06a32-e715-4a13-884e-443a85003b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:49.813085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef06a32-e715-4a13-884e-443a85003b4c None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:49.813085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef06a32-e715-4a13-884e-443a85003b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:49.813249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef06a32-e715-4a13-884e-443a85003b4c None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:49.813785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef06a32-e715-4a13-884e-443a85003b4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:49.815287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef06a32-e715-4a13-884e-443a85003b4c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:49.841527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 236/470] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:54:49 2022] POST /identity/v3/users => generated 248 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:54:49.842342 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef06a32-e715-4a13-884e-443a85003b4c None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:49.900122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 235/471] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:49.977925 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef06a32-e715-4a13-884e-443a85003b4c None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:50.016130 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e630b3-d9ba-4b67-9857-516f5a46954d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:50.016769 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e630b3-d9ba-4b67-9857-516f5a46954d None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:50.017056 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e630b3-d9ba-4b67-9857-516f5a46954d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:50.017382 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e630b3-d9ba-4b67-9857-516f5a46954d None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:50.018057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e630b3-d9ba-4b67-9857-516f5a46954d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:50.019653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e630b3-d9ba-4b67-9857-516f5a46954d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:50.046415 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 237/472] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:54:49 2022] POST /identity/v3/users => generated 248 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:54:50.192509 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb236818-15f7-41bc-9511-94457def7f8a None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:50.192790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb236818-15f7-41bc-9511-94457def7f8a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:50.193063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb236818-15f7-41bc-9511-94457def7f8a None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:50.194442 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 236/473] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:50.201622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50ec66-2214-435a-9e2c-e551cc97f8fc None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:50.201925 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50ec66-2214-435a-9e2c-e551cc97f8fc None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:50.202173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50ec66-2214-435a-9e2c-e551cc97f8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:50.220401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb236818-15f7-41bc-9511-94457def7f8a None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:50.225455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-ae50ec66-2214-435a-9e2c-e551cc97f8fc None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:50.226691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ae50ec66-2214-435a-9e2c-e551cc97f8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:50.230992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:50.260678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:50.317092 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 238/474] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:50.357416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb236818-15f7-41bc-9511-94457def7f8a None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:50.394222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eaa8ae-e71b-4d9a-a2e6-6f0f854391e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:50.395669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaa8ae-e71b-4d9a-a2e6-6f0f854391e2 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:50.396155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaa8ae-e71b-4d9a-a2e6-6f0f854391e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:50.396597 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eaa8ae-e71b-4d9a-a2e6-6f0f854391e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:50.408086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eaa8ae-e71b-4d9a-a2e6-6f0f854391e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:50.409875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eaa8ae-e71b-4d9a-a2e6-6f0f854391e2 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-b1eaa8ae-e71b-4d9a-a2e6-6f0f854391e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:54:50.429622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:50.430618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:54:50.431494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:54:50.431494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:50.431494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:50.433382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 237/475] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:54:50.439521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50ec66-2214-435a-9e2c-e551cc97f8fc None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:50.553154 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50ec66-2214-435a-9e2c-e551cc97f8fc None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:50.589895 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de1e0e-5ab7-461f-897c-f80ce8e47b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:50.590803 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de1e0e-5ab7-461f-897c-f80ce8e47b90 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:50.591224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de1e0e-5ab7-461f-897c-f80ce8e47b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:50.591667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de1e0e-5ab7-461f-897c-f80ce8e47b90 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:50.592465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de1e0e-5ab7-461f-897c-f80ce8e47b90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:50.592912 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de1e0e-5ab7-461f-897c-f80ce8e47b90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:50.594486 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de1e0e-5ab7-461f-897c-f80ce8e47b90 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:50.603817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:50.608125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 239/476] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:50.613996 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eaa8ae-e71b-4d9a-a2e6-6f0f854391e2 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:50.687438 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de1e0e-5ab7-461f-897c-f80ce8e47b90 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:50.706437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:50.732894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eaa8ae-e71b-4d9a-a2e6-6f0f854391e2 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:50.773166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343ff91-ada3-4e11-af88-8f10bbdfbca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:50.773792 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343ff91-ada3-4e11-af88-8f10bbdfbca7 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:50.774327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343ff91-ada3-4e11-af88-8f10bbdfbca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:50.774605 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343ff91-ada3-4e11-af88-8f10bbdfbca7 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:50.775281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343ff91-ada3-4e11-af88-8f10bbdfbca7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:50.777298 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343ff91-ada3-4e11-af88-8f10bbdfbca7 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:50.809876 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 238/477] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:54:50 2022] POST /identity/v3/users => generated 249 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:54:50.870743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de1e0e-5ab7-461f-897c-f80ce8e47b90 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:50.908709 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b75cf4-c4cc-4d40-aaff-fb28ec5853e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:50.909514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b75cf4-c4cc-4d40-aaff-fb28ec5853e2 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:50.909800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b75cf4-c4cc-4d40-aaff-fb28ec5853e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:50.910073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b75cf4-c4cc-4d40-aaff-fb28ec5853e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:50.910823 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b75cf4-c4cc-4d40-aaff-fb28ec5853e2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:50.912258 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b75cf4-c4cc-4d40-aaff-fb28ec5853e2 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:50.929104 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.sql.core [None req-e4b75cf4-c4cc-4d40-aaff-fb28ec5853e2 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 15:54:50.929104 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 08 15:54:50.929104 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [parameters: {'id': 'a125e683996b49b0a2a1b076a816f59f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 08 15:54:50.929104 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81526) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 15:54:50.950659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-e4b75cf4-c4cc-4d40-aaff-fb28ec5853e2 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 08 15:54:50.950659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:50.950659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 15:54:50.950659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 15:54:50.950659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 15:54:50.950659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 15:54:50.950659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 15:54:50.950659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 15:54:50.950659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 15:54:50.950659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application conn.query(q) Sep 08 15:54:50.950659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 15:54:50.950659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 15:54:50.950659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 15:54:50.950659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application result.read() Sep 08 15:54:50.950659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return ref.to_dict() Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 15:54:50.951622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application next(self.gen) Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application yield resource Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application next(self.gen) Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise self.value Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application session.commit() Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 15:54:50.953205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self.session.flush() Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application compat.raise_( Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 15:54:50.954057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 15:54:50.954914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application conn.query(q) Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application result.read() Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 15:54:50.955818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [parameters: {'id': 'a125e683996b49b0a2a1b076a816f59f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:50.956653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 08 15:54:50.957447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:50.958262 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 240/478] 10.222.0.27 () {60 vars in 1131 bytes} [Thu Sep 8 15:54:50 2022] POST /identity/v3/roles => generated 144 bytes in 265 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:50.965740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343ff91-ada3-4e11-af88-8f10bbdfbca7 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:51.103355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343ff91-ada3-4e11-af88-8f10bbdfbca7 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:51.133286 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:51.138641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5bbb8-6c22-4de7-9fd3-26d6f64c0655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:51.139280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5bbb8-6c22-4de7-9fd3-26d6f64c0655 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:51.139597 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5bbb8-6c22-4de7-9fd3-26d6f64c0655 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:51.139893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5bbb8-6c22-4de7-9fd3-26d6f64c0655 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:51.153016 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5bbb8-6c22-4de7-9fd3-26d6f64c0655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-c5d5bbb8-6c22-4de7-9fd3-26d6f64c0655 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:51.158371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:51.160402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:51.161429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:51.161429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:51.161429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:51.161429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:51.161429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:51.161429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:51.161429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:51.161429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:51.161429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:51.161429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:51.161429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 15:54:51.161429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:51.161429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 239/479] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:54:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:54:51.173530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b75cf4-c4cc-4d40-aaff-fb28ec5853e2 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:51.263208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:51.315262 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b75cf4-c4cc-4d40-aaff-fb28ec5853e2 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:51.362519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5ce91-e8a4-4c3c-a65d-3a64e122a7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:51.363131 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5ce91-e8a4-4c3c-a65d-3a64e122a7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:51.363410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5ce91-e8a4-4c3c-a65d-3a64e122a7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:51.363703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5ce91-e8a4-4c3c-a65d-3a64e122a7f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:51.364372 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5ce91-e8a4-4c3c-a65d-3a64e122a7f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:51.364668 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5ce91-e8a4-4c3c-a65d-3a64e122a7f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:51.366057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5ce91-e8a4-4c3c-a65d-3a64e122a7f2 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:51.379059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 241/480] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:52.335635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82606a-065d-4a3f-a747-e24b16f8070f None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:52.338727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82606a-065d-4a3f-a747-e24b16f8070f None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:52.338727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82606a-065d-4a3f-a747-e24b16f8070f None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:52.339311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 240/481] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:52.343919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf64d30-6712-4ad6-9d34-f06b43c81295 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:52.344388 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf64d30-6712-4ad6-9d34-f06b43c81295 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:52.344535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf64d30-6712-4ad6-9d34-f06b43c81295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:52.372460 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-ebf64d30-6712-4ad6-9d34-f06b43c81295 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:52.373667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ebf64d30-6712-4ad6-9d34-f06b43c81295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:52.443915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 242/482] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:52.696557 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537b475-dacf-4450-8ba0-eebe807e1198 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:52.696740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537b475-dacf-4450-8ba0-eebe807e1198 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:52.697016 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9537b475-dacf-4450-8ba0-eebe807e1198 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:52.698625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 241/483] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:52.703969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b07543-5848-4039-9ba4-8e85a830b1e1 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:52.704245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b07543-5848-4039-9ba4-8e85a830b1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:52.704514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b07543-5848-4039-9ba4-8e85a830b1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:52.725671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcc2ca-d96f-41e4-89d6-d96c50037804 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:52.725946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcc2ca-d96f-41e4-89d6-d96c50037804 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:52.726216 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bcc2ca-d96f-41e4-89d6-d96c50037804 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:52.727890 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 242/484] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:54:52.732753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef0abe-6320-4e48-bd0b-83beed19be59 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:52.733036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef0abe-6320-4e48-bd0b-83beed19be59 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:52.733311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef0abe-6320-4e48-bd0b-83beed19be59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:52.736017 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-53b07543-5848-4039-9ba4-8e85a830b1e1 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:52.737519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-53b07543-5848-4039-9ba4-8e85a830b1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:52.764902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-cdef0abe-6320-4e48-bd0b-83beed19be59 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:52.767369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-cdef0abe-6320-4e48-bd0b-83beed19be59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:52.832336 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:52.859619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 243/485] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:52.860652 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b07543-5848-4039-9ba4-8e85a830b1e1 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:52.870806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 243/486] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:52.971295 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b07543-5848-4039-9ba4-8e85a830b1e1 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:53.009203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2dcde-a221-4108-9f11-5994acb9f1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:53.009863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2dcde-a221-4108-9f11-5994acb9f1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:53.010132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2dcde-a221-4108-9f11-5994acb9f1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:53.010395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc2dcde-a221-4108-9f11-5994acb9f1c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:53.021802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc2dcde-a221-4108-9f11-5994acb9f1c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-2dc2dcde-a221-4108-9f11-5994acb9f1c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:53.026682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:53.027603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:53.028404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:53.028404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:53.028404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:53.028404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:53.028404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:53.028404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:53.028404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:53.028404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:53.028404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:53.028404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:53.028404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:54:53.028404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:53.028404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 244/487] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:54:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:53.033815 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef0abe-6320-4e48-bd0b-83beed19be59 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:53.130309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2dcde-a221-4108-9f11-5994acb9f1c5 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:53.187295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdef0abe-6320-4e48-bd0b-83beed19be59 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:53.228001 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:53.229952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:53.254970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a369e-2042-45f0-bae7-28b4c5d54cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:53.255943 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a369e-2042-45f0-bae7-28b4c5d54cdb None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:53.256575 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a369e-2042-45f0-bae7-28b4c5d54cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:53.257033 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9a369e-2042-45f0-bae7-28b4c5d54cdb None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:53.257921 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a369e-2042-45f0-bae7-28b4c5d54cdb None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:53.258407 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a369e-2042-45f0-bae7-28b4c5d54cdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:53.260002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9a369e-2042-45f0-bae7-28b4c5d54cdb None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:53.271834 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 244/488] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:54:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:53.272945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a369e-2042-45f0-bae7-28b4c5d54cdb None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:53.301745 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc2dcde-a221-4108-9f11-5994acb9f1c5 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:53.361518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf5f79-3c67-4ea4-91b3-7b80b6d4e8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:53.361518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf5f79-3c67-4ea4-91b3-7b80b6d4e8ac None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:53.361518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf5f79-3c67-4ea4-91b3-7b80b6d4e8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:53.361518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf5f79-3c67-4ea4-91b3-7b80b6d4e8ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:53.381333 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9a369e-2042-45f0-bae7-28b4c5d54cdb None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:53.384305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf5f79-3c67-4ea4-91b3-7b80b6d4e8ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-8edf5f79-3c67-4ea4-91b3-7b80b6d4e8ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:53.389416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:53.390494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:53.391530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:53.391530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:53.391530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:53.391530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:53.391530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:53.391530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:53.391530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:53.391530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:53.391530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:53.391530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:53.391530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:54:53.391530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:53.401368 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 245/489] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:54:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:54:53.402516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf5f79-3c67-4ea4-91b3-7b80b6d4e8ac None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:53.433845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c5324-822e-492a-a9e5-f5e3d98a419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:53.434911 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c5324-822e-492a-a9e5-f5e3d98a419d None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:53.436352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c5324-822e-492a-a9e5-f5e3d98a419d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:53.436835 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c5324-822e-492a-a9e5-f5e3d98a419d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:53.467547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c5324-822e-492a-a9e5-f5e3d98a419d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-6f3c5324-822e-492a-a9e5-f5e3d98a419d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:53.473978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:53.475120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:53.476163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:53.476163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:53.476163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:53.476163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:53.476163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:53.476163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:53.476163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:53.476163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:53.476163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:53.476163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:53.476163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:54:53.476163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:53.488165 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 245/490] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:54:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:53.489404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c5324-822e-492a-a9e5-f5e3d98a419d None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:53.569371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:53.596456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf5f79-3c67-4ea4-91b3-7b80b6d4e8ac None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:53.605682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:53.623553 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c5324-822e-492a-a9e5-f5e3d98a419d None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:53.638725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58063c1-faf4-4ee7-b6f8-161a5065c0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:53.639506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58063c1-faf4-4ee7-b6f8-161a5065c0aa None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:53.639852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58063c1-faf4-4ee7-b6f8-161a5065c0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:53.640186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58063c1-faf4-4ee7-b6f8-161a5065c0aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:53.654445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58063c1-faf4-4ee7-b6f8-161a5065c0aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:53.656133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58063c1-faf4-4ee7-b6f8-161a5065c0aa None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:53.660097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713fb48-40e0-444a-aa46-71ae44c1c7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:53.660976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713fb48-40e0-444a-aa46-71ae44c1c7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:53.661446 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713fb48-40e0-444a-aa46-71ae44c1c7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:53.661907 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713fb48-40e0-444a-aa46-71ae44c1c7b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:53.662769 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713fb48-40e0-444a-aa46-71ae44c1c7b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:53.663261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713fb48-40e0-444a-aa46-71ae44c1c7b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:53.664965 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713fb48-40e0-444a-aa46-71ae44c1c7b5 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-b58063c1-faf4-4ee7-b6f8-161a5065c0aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:54:53.674245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:53.675431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:54:53.676322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:54:53.676322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:53.676322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:53.676481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 246/491] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:54:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:53.677380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713fb48-40e0-444a-aa46-71ae44c1c7b5 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:53.678229 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 246/492] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:53.679624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee6096-98d9-414f-b84b-545a841aac47 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:53.680141 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee6096-98d9-414f-b84b-545a841aac47 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:53.680604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee6096-98d9-414f-b84b-545a841aac47 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:53.686547 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 247/493] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:53.687736 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee6096-98d9-414f-b84b-545a841aac47 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:53.787438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713fb48-40e0-444a-aa46-71ae44c1c7b5 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:53.794916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:53.801022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee6096-98d9-414f-b84b-545a841aac47 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:53.826849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ecb52d-c4d0-47bc-aee7-1423296d2c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:53.827807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecb52d-c4d0-47bc-aee7-1423296d2c38 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:53.828228 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecb52d-c4d0-47bc-aee7-1423296d2c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:53.828504 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ecb52d-c4d0-47bc-aee7-1423296d2c38 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:53.829230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecb52d-c4d0-47bc-aee7-1423296d2c38 None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:53.829511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecb52d-c4d0-47bc-aee7-1423296d2c38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:53.830920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ecb52d-c4d0-47bc-aee7-1423296d2c38 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:53.838290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e8086-c707-4a52-9964-fa7574934342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:53.838923 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e8086-c707-4a52-9964-fa7574934342 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:53.839194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e8086-c707-4a52-9964-fa7574934342 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:53.839535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e8086-c707-4a52-9964-fa7574934342 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:53.840043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 247/494] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:54:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:53.840043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ecb52d-c4d0-47bc-aee7-1423296d2c38 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:53.855223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e8086-c707-4a52-9964-fa7574934342 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:53.856752 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e8086-c707-4a52-9964-fa7574934342 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-de4e8086-c707-4a52-9964-fa7574934342 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:54:53.871861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:53.872797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:54:53.873653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:54:53.873653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:53.873653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:53.873653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 248/495] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:53.874386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89e9c3-9e10-49cd-b944-54a341ad0b12 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:53.874614 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89e9c3-9e10-49cd-b944-54a341ad0b12 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:53.874883 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89e9c3-9e10-49cd-b944-54a341ad0b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:53.893327 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-de89e9c3-9e10-49cd-b944-54a341ad0b12 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:53.894886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-de89e9c3-9e10-49cd-b944-54a341ad0b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:53.898548 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:53.956414 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ecb52d-c4d0-47bc-aee7-1423296d2c38 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:53.967658 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 249/496] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:53.967658 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89e9c3-9e10-49cd-b944-54a341ad0b12 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:54.002345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e67b2a-bc14-42fa-8aac-669086d3bf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:54.003269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e67b2a-bc14-42fa-8aac-669086d3bf2d None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:54.003711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e67b2a-bc14-42fa-8aac-669086d3bf2d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:54.004145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e67b2a-bc14-42fa-8aac-669086d3bf2d None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:54.005999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e67b2a-bc14-42fa-8aac-669086d3bf2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:54.006453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e67b2a-bc14-42fa-8aac-669086d3bf2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:54.008012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e67b2a-bc14-42fa-8aac-669086d3bf2d None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:54.023878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 248/497] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:54.024983 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e67b2a-bc14-42fa-8aac-669086d3bf2d None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:54.064040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:54.088858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89e9c3-9e10-49cd-b944-54a341ad0b12 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:54.125960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02c2db-4fd9-4dd2-920f-2596d784fd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:54.126844 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02c2db-4fd9-4dd2-920f-2596d784fd13 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:54.127361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02c2db-4fd9-4dd2-920f-2596d784fd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:54.127843 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02c2db-4fd9-4dd2-920f-2596d784fd13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:54.137235 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:54.140976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02c2db-4fd9-4dd2-920f-2596d784fd13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:54.142677 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02c2db-4fd9-4dd2-920f-2596d784fd13 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:54.143849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e67b2a-bc14-42fa-8aac-669086d3bf2d None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-db02c2db-4fd9-4dd2-920f-2596d784fd13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:54:54.159941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:54.161076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:54:54.162120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:54:54.162120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:54.162120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:54.163882 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 250/498] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:54.164887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02c2db-4fd9-4dd2-920f-2596d784fd13 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:54.188550 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feed8fd-0121-4bbd-8ac5-ff3e2d14c619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:54.189315 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feed8fd-0121-4bbd-8ac5-ff3e2d14c619 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:54.189631 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feed8fd-0121-4bbd-8ac5-ff3e2d14c619 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:54.189899 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feed8fd-0121-4bbd-8ac5-ff3e2d14c619 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:54.190643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feed8fd-0121-4bbd-8ac5-ff3e2d14c619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:54.190940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feed8fd-0121-4bbd-8ac5-ff3e2d14c619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:54.192403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feed8fd-0121-4bbd-8ac5-ff3e2d14c619 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:54.205360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 249/499] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:54.206077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feed8fd-0121-4bbd-8ac5-ff3e2d14c619 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:54.275029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02c2db-4fd9-4dd2-920f-2596d784fd13 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:54.301332 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:54.312833 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43083a8c-4086-4457-8443-04d376950c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:54.313506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43083a8c-4086-4457-8443-04d376950c25 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:54.313776 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43083a8c-4086-4457-8443-04d376950c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:54.314405 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43083a8c-4086-4457-8443-04d376950c25 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:54.317119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feed8fd-0121-4bbd-8ac5-ff3e2d14c619 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:54.324218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43083a8c-4086-4457-8443-04d376950c25 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:54.325722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43083a8c-4086-4457-8443-04d376950c25 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-43083a8c-4086-4457-8443-04d376950c25 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:54:54.339981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:54.341165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:54.347249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:54.347249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:54.347249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:54:54.347249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:54:54.347249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:54:54.347249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:54:54.347249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:54:54.347249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:54:54.347249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:54:54.347249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:54.347249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 251/500] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Sep 8 15:54:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:54.347249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43083a8c-4086-4457-8443-04d376950c25 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:54.364836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece0b5e-0dfe-496a-96ce-5dcc335735eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:54.365626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece0b5e-0dfe-496a-96ce-5dcc335735eb None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:54.365874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece0b5e-0dfe-496a-96ce-5dcc335735eb None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:54.366208 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece0b5e-0dfe-496a-96ce-5dcc335735eb None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:54.366908 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece0b5e-0dfe-496a-96ce-5dcc335735eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:54.367296 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece0b5e-0dfe-496a-96ce-5dcc335735eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:54.368754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece0b5e-0dfe-496a-96ce-5dcc335735eb None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:54.381640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 250/501] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:54.383543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece0b5e-0dfe-496a-96ce-5dcc335735eb None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:54.409002 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:54.458888 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43083a8c-4086-4457-8443-04d376950c25 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:54.492862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f8d26-e09d-4116-bb7f-5905a24cee0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:54.493493 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f8d26-e09d-4116-bb7f-5905a24cee0b None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:54.493762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f8d26-e09d-4116-bb7f-5905a24cee0b None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:54.494034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f8d26-e09d-4116-bb7f-5905a24cee0b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:54.495888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece0b5e-0dfe-496a-96ce-5dcc335735eb None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:54.501350 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f8d26-e09d-4116-bb7f-5905a24cee0b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:54.503178 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f8d26-e09d-4116-bb7f-5905a24cee0b None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-896f8d26-e09d-4116-bb7f-5905a24cee0b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:54:54.513985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:54.514886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:54.515866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:54.515866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:54.515866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:54:54.515866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:54:54.515866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:54:54.515866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:54:54.515866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:54:54.515866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:54:54.515866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 15:54:54.515866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:54.515866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 252/502] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:54.515866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f8d26-e09d-4116-bb7f-5905a24cee0b None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:54.533976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652f671-c5f9-4326-97f2-ac50b3df672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:54.534639 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652f671-c5f9-4326-97f2-ac50b3df672a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:54.534910 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652f671-c5f9-4326-97f2-ac50b3df672a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:54.535178 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652f671-c5f9-4326-97f2-ac50b3df672a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:54.545711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652f671-c5f9-4326-97f2-ac50b3df672a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-3652f671-c5f9-4326-97f2-ac50b3df672a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:54.550496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:54.551568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:54.552538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:54.552538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:54.552538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:54.552538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:54.552538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:54.552538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:54.552538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:54.552538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:54.552538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:54.552538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:54.552538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:54:54.552538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:54.552538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 251/503] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:54:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:54.552538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652f671-c5f9-4326-97f2-ac50b3df672a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:54.565931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:54.615852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f8d26-e09d-4116-bb7f-5905a24cee0b None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:54.628066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:54.649274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe415740-1597-498a-bce9-e14f733e366e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:54.649873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe415740-1597-498a-bce9-e14f733e366e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:54.650146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe415740-1597-498a-bce9-e14f733e366e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:54.650411 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe415740-1597-498a-bce9-e14f733e366e None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:54.651132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe415740-1597-498a-bce9-e14f733e366e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:54.651427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe415740-1597-498a-bce9-e14f733e366e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:54.652958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe415740-1597-498a-bce9-e14f733e366e None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:54.655413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652f671-c5f9-4326-97f2-ac50b3df672a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:54.687997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44394b-67df-48d2-8cb3-b1ca38f69f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:54.688208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 253/504] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Sep 8 15:54:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:54.688628 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44394b-67df-48d2-8cb3-b1ca38f69f88 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:54.688902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44394b-67df-48d2-8cb3-b1ca38f69f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:54.689171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44394b-67df-48d2-8cb3-b1ca38f69f88 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:54.689419 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4ee04-7ac1-4393-862c-a7ff2567e68c None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:54.689695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4ee04-7ac1-4393-862c-a7ff2567e68c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:54.689959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4ee04-7ac1-4393-862c-a7ff2567e68c None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:54.691346 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 254/505] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:54.691902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4ee04-7ac1-4393-862c-a7ff2567e68c None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:54.697532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44394b-67df-48d2-8cb3-b1ca38f69f88 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:54.698996 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc44394b-67df-48d2-8cb3-b1ca38f69f88 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-fc44394b-67df-48d2-8cb3-b1ca38f69f88 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:54:54.711912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:54.713363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:54.714279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:54.714279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:54.714279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:54:54.714279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:54:54.714279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:54:54.714279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:54:54.714279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:54:54.714279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:54:54.714279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 15:54:54.714279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:54.714279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 252/506] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:54:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:54.714835 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44394b-67df-48d2-8cb3-b1ca38f69f88 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:54.778398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:54.794696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4ee04-7ac1-4393-862c-a7ff2567e68c None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:54.813503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc44394b-67df-48d2-8cb3-b1ca38f69f88 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:54.827221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75305a0-cb4c-4aff-9ead-3b737c25a0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:54.827797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75305a0-cb4c-4aff-9ead-3b737c25a0ce None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:54.828075 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75305a0-cb4c-4aff-9ead-3b737c25a0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:54.828342 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75305a0-cb4c-4aff-9ead-3b737c25a0ce None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:54.829071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75305a0-cb4c-4aff-9ead-3b737c25a0ce None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:54.829371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75305a0-cb4c-4aff-9ead-3b737c25a0ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:54.830926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75305a0-cb4c-4aff-9ead-3b737c25a0ce None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:54.845819 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586d575-0789-4825-a21a-c6308e0f7e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:54.846447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586d575-0789-4825-a21a-c6308e0f7e46 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:54.846717 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586d575-0789-4825-a21a-c6308e0f7e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:54.846980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586d575-0789-4825-a21a-c6308e0f7e46 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:54.847668 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0586d575-0789-4825-a21a-c6308e0f7e46 None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:54.847987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0586d575-0789-4825-a21a-c6308e0f7e46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:54.848088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 255/507] 10.222.0.27 () {56 vars in 1137 bytes} [Thu Sep 8 15:54:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:54.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75305a0-cb4c-4aff-9ead-3b737c25a0ce None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:54.849356 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0586d575-0789-4825-a21a-c6308e0f7e46 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:54.858033 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 253/508] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:54:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:54.859198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51c9c2-5189-45fe-93f6-3e9159dba2a5 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:54.859500 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51c9c2-5189-45fe-93f6-3e9159dba2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:54.859771 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51c9c2-5189-45fe-93f6-3e9159dba2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:54.877873 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-5a51c9c2-5189-45fe-93f6-3e9159dba2a5 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:54.879055 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5a51c9c2-5189-45fe-93f6-3e9159dba2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:54.879152 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:54.943126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 254/509] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:54:54.944001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51c9c2-5189-45fe-93f6-3e9159dba2a5 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:54.968823 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75305a0-cb4c-4aff-9ead-3b737c25a0ce None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:55.002544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:55.008742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1b220-70d5-446c-8c0c-c757a7a04c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:55.009355 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1b220-70d5-446c-8c0c-c757a7a04c41 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:55.009656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1b220-70d5-446c-8c0c-c757a7a04c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:55.009959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1b220-70d5-446c-8c0c-c757a7a04c41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:55.046270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51c9c2-5189-45fe-93f6-3e9159dba2a5 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:55.079223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1b220-70d5-446c-8c0c-c757a7a04c41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:55.080609 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3beb2-1ce7-4c24-8625-4e6b90168430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:55.080761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb1b220-70d5-446c-8c0c-c757a7a04c41 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:55.081321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3beb2-1ce7-4c24-8625-4e6b90168430 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:55.081592 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3beb2-1ce7-4c24-8625-4e6b90168430 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:55.081855 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f3beb2-1ce7-4c24-8625-4e6b90168430 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:55.082588 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3beb2-1ce7-4c24-8625-4e6b90168430 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:55.082885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3beb2-1ce7-4c24-8625-4e6b90168430 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:55.084320 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f3beb2-1ce7-4c24-8625-4e6b90168430 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:55.111571 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:55.127200 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 255/510] 10.222.0.27 () {56 vars in 1137 bytes} [Thu Sep 8 15:54:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:55.127200 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3beb2-1ce7-4c24-8625-4e6b90168430 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:55.192144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 256/511] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:55.193067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1b220-70d5-446c-8c0c-c757a7a04c41 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:55.236554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f3beb2-1ce7-4c24-8625-4e6b90168430 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:55.253255 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:55.270966 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5058ed-2301-48f6-9686-1538cd01a940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:55.271608 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5058ed-2301-48f6-9686-1538cd01a940 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:55.271928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5058ed-2301-48f6-9686-1538cd01a940 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:55.272198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5058ed-2301-48f6-9686-1538cd01a940 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:55.291247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb1b220-70d5-446c-8c0c-c757a7a04c41 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:55.317169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:55.325336 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc567450-52e9-49fd-93b9-d0ca37da38fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:55.325932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc567450-52e9-49fd-93b9-d0ca37da38fe None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:55.326234 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc567450-52e9-49fd-93b9-d0ca37da38fe None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:55.326498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc567450-52e9-49fd-93b9-d0ca37da38fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:55.336260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc567450-52e9-49fd-93b9-d0ca37da38fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:55.337604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5058ed-2301-48f6-9686-1538cd01a940 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:55.338112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc567450-52e9-49fd-93b9-d0ca37da38fe None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:55.339066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5058ed-2301-48f6-9686-1538cd01a940 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-dc567450-52e9-49fd-93b9-d0ca37da38fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:54:55.352208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:55.353316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:54:55.354330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:54:55.354330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:55.354330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:55.354330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 257/512] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:55.354330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc567450-52e9-49fd-93b9-d0ca37da38fe None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:55.438184 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 256/513] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:55.439021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5058ed-2301-48f6-9686-1538cd01a940 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:55.453159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc567450-52e9-49fd-93b9-d0ca37da38fe None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:55.460656 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:55.488440 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3bf80-a6a5-475d-9a85-071825ae0c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:55.489053 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3bf80-a6a5-475d-9a85-071825ae0c1d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:55.489326 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3bf80-a6a5-475d-9a85-071825ae0c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:55.489595 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3bf80-a6a5-475d-9a85-071825ae0c1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:55.536202 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:55.542907 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5058ed-2301-48f6-9686-1538cd01a940 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:55.556007 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3bf80-a6a5-475d-9a85-071825ae0c1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:55.557483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3bf80-a6a5-475d-9a85-071825ae0c1d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:55.576295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7e6cd-c190-43bb-80cf-9d56e3a166c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:55.576928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7e6cd-c190-43bb-80cf-9d56e3a166c9 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:55.577204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7e6cd-c190-43bb-80cf-9d56e3a166c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:55.577474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec7e6cd-c190-43bb-80cf-9d56e3a166c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:55.588190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7e6cd-c190-43bb-80cf-9d56e3a166c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:55.589573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec7e6cd-c190-43bb-80cf-9d56e3a166c9 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-eec7e6cd-c190-43bb-80cf-9d56e3a166c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:54:55.604275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:55.605316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:54:55.606198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:54:55.606198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:55.606198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:55.606198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 257/514] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:55.606198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7e6cd-c190-43bb-80cf-9d56e3a166c9 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:55.656725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 258/515] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:55.657490 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3bf80-a6a5-475d-9a85-071825ae0c1d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:55.674070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:55.705374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec7e6cd-c190-43bb-80cf-9d56e3a166c9 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:55.736081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:55.738466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294df4b-bda6-4c2f-97b1-af477d17f404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:55.739013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294df4b-bda6-4c2f-97b1-af477d17f404 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:55.739288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294df4b-bda6-4c2f-97b1-af477d17f404 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:55.739590 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294df4b-bda6-4c2f-97b1-af477d17f404 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:55.749958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294df4b-bda6-4c2f-97b1-af477d17f404 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:55.751406 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c294df4b-bda6-4c2f-97b1-af477d17f404 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:55.755408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3bf80-a6a5-475d-9a85-071825ae0c1d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-c294df4b-bda6-4c2f-97b1-af477d17f404 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:54:55.765280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:55.766188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:54:55.767040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:54:55.767040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:55.767040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:55.767040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 258/516] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:55.767263 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294df4b-bda6-4c2f-97b1-af477d17f404 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:55.794880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f2b9c-1fff-4f46-8807-492c719fcf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:55.795518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f2b9c-1fff-4f46-8807-492c719fcf2d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:55.795797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f2b9c-1fff-4f46-8807-492c719fcf2d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:55.796062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f2b9c-1fff-4f46-8807-492c719fcf2d None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:55.796706 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f2b9c-1fff-4f46-8807-492c719fcf2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:55.797010 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f2b9c-1fff-4f46-8807-492c719fcf2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:55.798432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f2b9c-1fff-4f46-8807-492c719fcf2d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:55.809389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 259/517] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:55.810201 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f2b9c-1fff-4f46-8807-492c719fcf2d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:55.864501 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c294df4b-bda6-4c2f-97b1-af477d17f404 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:55.886307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:55.897722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4561c-e7e4-4b63-8fe2-970048b69c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:55.898360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4561c-e7e4-4b63-8fe2-970048b69c84 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:55.898641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4561c-e7e4-4b63-8fe2-970048b69c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:55.898913 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4561c-e7e4-4b63-8fe2-970048b69c84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:55.906688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f2b9c-1fff-4f46-8807-492c719fcf2d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:55.909361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4561c-e7e4-4b63-8fe2-970048b69c84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:55.910830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4561c-e7e4-4b63-8fe2-970048b69c84 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-a4f4561c-e7e4-4b63-8fe2-970048b69c84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:54:55.924767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:55.925657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:54:55.926483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:54:55.926483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:55.926483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:55.926483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 259/518] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:55.926705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4561c-e7e4-4b63-8fe2-970048b69c84 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:55.944175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127abb6-eb78-4d12-bb8f-590e870268a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:55.944816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127abb6-eb78-4d12-bb8f-590e870268a1 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:55.945087 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127abb6-eb78-4d12-bb8f-590e870268a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:55.945350 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127abb6-eb78-4d12-bb8f-590e870268a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:55.945995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127abb6-eb78-4d12-bb8f-590e870268a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:55.946288 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127abb6-eb78-4d12-bb8f-590e870268a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:55.947669 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127abb6-eb78-4d12-bb8f-590e870268a1 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:55.956216 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:55.964575 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 260/519] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:55.964575 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127abb6-eb78-4d12-bb8f-590e870268a1 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:56.024771 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4561c-e7e4-4b63-8fe2-970048b69c84 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:56.057059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd332126-8c86-4bb7-97f3-ce9910e0078c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:56.057692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd332126-8c86-4bb7-97f3-ce9910e0078c None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:56.057961 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd332126-8c86-4bb7-97f3-ce9910e0078c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:56.058101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127abb6-eb78-4d12-bb8f-590e870268a1 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:56.058226 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd332126-8c86-4bb7-97f3-ce9910e0078c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:56.069147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd332126-8c86-4bb7-97f3-ce9910e0078c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:56.070666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd332126-8c86-4bb7-97f3-ce9910e0078c None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-cd332126-8c86-4bb7-97f3-ce9910e0078c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:54:56.084418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:56.085492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:54:56.086372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:54:56.086372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:56.086372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:56.086372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 260/520] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:56.086596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd332126-8c86-4bb7-97f3-ce9910e0078c None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:56.095674 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7691d9-ca75-4d14-8100-f5386f56999a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:56.096316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7691d9-ca75-4d14-8100-f5386f56999a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:56.096636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7691d9-ca75-4d14-8100-f5386f56999a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:56.096912 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7691d9-ca75-4d14-8100-f5386f56999a None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:56.097560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7691d9-ca75-4d14-8100-f5386f56999a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:56.097860 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7691d9-ca75-4d14-8100-f5386f56999a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:56.099240 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7691d9-ca75-4d14-8100-f5386f56999a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:56.110745 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 261/521] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:56.111679 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7691d9-ca75-4d14-8100-f5386f56999a None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:56.121127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:56.170277 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:56.194443 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd332126-8c86-4bb7-97f3-ce9910e0078c None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:56.214062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7691d9-ca75-4d14-8100-f5386f56999a None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:56.227164 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d6f50-70ec-415f-80fb-5f5c1b9ed8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:56.227820 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d6f50-70ec-415f-80fb-5f5c1b9ed8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:56.228091 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d6f50-70ec-415f-80fb-5f5c1b9ed8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:56.228357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d6f50-70ec-415f-80fb-5f5c1b9ed8d3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:56.235977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d6f50-70ec-415f-80fb-5f5c1b9ed8d3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:56.237454 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395d6f50-70ec-415f-80fb-5f5c1b9ed8d3 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:56.247419 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9271b0-0aec-48af-af17-36b7cf781a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:56.248054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9271b0-0aec-48af-af17-36b7cf781a7b None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-395d6f50-70ec-415f-80fb-5f5c1b9ed8d3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:54:56.248265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:56.249261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:56.250171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:56.250171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:56.250171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:54:56.250171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:54:56.250171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:54:56.250171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:54:56.250171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:54:56.250171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:54:56.250171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 15:54:56.250171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:56.251284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9271b0-0aec-48af-af17-36b7cf781a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:56.251284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9271b0-0aec-48af-af17-36b7cf781a7b None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:56.251284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9271b0-0aec-48af-af17-36b7cf781a7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:56.251499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 261/522] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:54:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:56.251572 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9271b0-0aec-48af-af17-36b7cf781a7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:56.251639 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d6f50-70ec-415f-80fb-5f5c1b9ed8d3 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:56.251705 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9271b0-0aec-48af-af17-36b7cf781a7b None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:56.263466 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 262/523] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:56.264572 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9271b0-0aec-48af-af17-36b7cf781a7b None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:56.332219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:56.348178 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d6f50-70ec-415f-80fb-5f5c1b9ed8d3 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:56.362775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9271b0-0aec-48af-af17-36b7cf781a7b None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:56.382329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:56.383291 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97e9c0-fd38-4bbd-b2a8-f3ebbcbe966d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:56.384055 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97e9c0-fd38-4bbd-b2a8-f3ebbcbe966d None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:56.384348 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97e9c0-fd38-4bbd-b2a8-f3ebbcbe966d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:56.384653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97e9c0-fd38-4bbd-b2a8-f3ebbcbe966d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:56.391268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97e9c0-fd38-4bbd-b2a8-f3ebbcbe966d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:56.392758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97e9c0-fd38-4bbd-b2a8-f3ebbcbe966d None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:56.396369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97e903-fd7e-4504-90cc-7b38873cfc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:56.396969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97e903-fd7e-4504-90cc-7b38873cfc79 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:56.397240 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97e903-fd7e-4504-90cc-7b38873cfc79 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:56.397507 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a97e903-fd7e-4504-90cc-7b38873cfc79 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:56.398167 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97e903-fd7e-4504-90cc-7b38873cfc79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:56.398465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97e903-fd7e-4504-90cc-7b38873cfc79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:56.399863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a97e903-fd7e-4504-90cc-7b38873cfc79 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:56.404387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-0f97e9c0-fd38-4bbd-b2a8-f3ebbcbe966d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:54:56.404387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:56.404387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:56.404387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:56.404387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:56.404387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:56.404387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:56.404387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:56.404387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:56.404387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:56.404387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:56.404387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:56.404387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:54:56.404387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:54:56.404387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:56.405249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:56.406100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:54:56.406100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:54:56.406100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:54:56.406100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:54:56.406100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:54:56.406100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:54:56.406100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:54:56.406100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:56.407592 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 262/524] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:56.409016 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97e9c0-fd38-4bbd-b2a8-f3ebbcbe966d None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:56.415903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 263/525] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:56.415903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97e903-fd7e-4504-90cc-7b38873cfc79 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:56.510293 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a97e903-fd7e-4504-90cc-7b38873cfc79 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:56.510945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97e9c0-fd38-4bbd-b2a8-f3ebbcbe966d None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:56.545302 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46fbb66-64e7-4f62-907f-56094af2dbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:56.545973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fbb66-64e7-4f62-907f-56094af2dbf4 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:56.546403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fbb66-64e7-4f62-907f-56094af2dbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:56.546691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fbb66-64e7-4f62-907f-56094af2dbf4 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:56.547377 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fbb66-64e7-4f62-907f-56094af2dbf4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:56.548838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fbb66-64e7-4f62-907f-56094af2dbf4 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:56.551497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049283b-c6d5-4f73-8544-dd883d5ea452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:56.552103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049283b-c6d5-4f73-8544-dd883d5ea452 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:56.552376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049283b-c6d5-4f73-8544-dd883d5ea452 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:56.552642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049283b-c6d5-4f73-8544-dd883d5ea452 None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:56.553398 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049283b-c6d5-4f73-8544-dd883d5ea452 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:56.553695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049283b-c6d5-4f73-8544-dd883d5ea452 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:56.555188 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049283b-c6d5-4f73-8544-dd883d5ea452 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:56.570439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 263/526] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:54:56 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:54:56.571258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46fbb66-64e7-4f62-907f-56094af2dbf4 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:56.573897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:56.574559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 264/527] 10.222.0.27 () {56 vars in 1139 bytes} [Thu Sep 8 15:54:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:56.581504 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049283b-c6d5-4f73-8544-dd883d5ea452 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:56.631628 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:56.692733 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46fbb66-64e7-4f62-907f-56094af2dbf4 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:56.697010 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049283b-c6d5-4f73-8544-dd883d5ea452 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:56.732015 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebf921-a7e4-451a-a24b-0376a0e754ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:56.732666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebf921-a7e4-451a-a24b-0376a0e754ac None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:56.732934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebf921-a7e4-451a-a24b-0376a0e754ac None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:56.733214 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebf921-a7e4-451a-a24b-0376a0e754ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:56.735046 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fd83c-55fa-40f4-a532-9c6982248e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:56.735903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fd83c-55fa-40f4-a532-9c6982248e71 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:56.736494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fd83c-55fa-40f4-a532-9c6982248e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:56.736779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fd83c-55fa-40f4-a532-9c6982248e71 None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:56.737540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fd83c-55fa-40f4-a532-9c6982248e71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:56.737845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fd83c-55fa-40f4-a532-9c6982248e71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:56.739306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fd83c-55fa-40f4-a532-9c6982248e71 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:56.741416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebf921-a7e4-451a-a24b-0376a0e754ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:56.742561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebf921-a7e4-451a-a24b-0376a0e754ac None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:56.754960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-a7ebf921-a7e4-451a-a24b-0376a0e754ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:54:56.754960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:56.754960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:56.754960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:56.754960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:56.754960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:56.754960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:56.754960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:56.754960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:56.754960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:56.754960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:56.754960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:56.754960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:54:56.754960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:54:56.754960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:56.756005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:56.756920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:54:56.756920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:54:56.756920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:54:56.756920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:54:56.756920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:54:56.756920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:54:56.756920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:54:56.756920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:56.756920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 264/528] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:56.757402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebf921-a7e4-451a-a24b-0376a0e754ac None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:56.758819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 265/529] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:54:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:56.760069 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fd83c-55fa-40f4-a532-9c6982248e71 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:56.828072 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:56.870079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:56.872993 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebf921-a7e4-451a-a24b-0376a0e754ac None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:56.874603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fd83c-55fa-40f4-a532-9c6982248e71 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:56.911304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25c899-ab4e-4175-b301-70dca056d844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:56.911676 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2c1a1-04cc-4dea-ae35-f0042740a14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:56.911977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25c899-ab4e-4175-b301-70dca056d844 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:56.912245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2c1a1-04cc-4dea-ae35-f0042740a14d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:56.912339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25c899-ab4e-4175-b301-70dca056d844 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:56.912519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2c1a1-04cc-4dea-ae35-f0042740a14d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:56.912682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25c899-ab4e-4175-b301-70dca056d844 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:56.912771 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2c1a1-04cc-4dea-ae35-f0042740a14d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:56.919764 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25c899-ab4e-4175-b301-70dca056d844 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:56.921228 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25c899-ab4e-4175-b301-70dca056d844 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:56.933215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-ee25c899-ab4e-4175-b301-70dca056d844 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:54:56.933215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:56.933215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:56.933215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:56.933215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:56.933215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:56.933215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:56.933215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:56.933215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:56.933215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:56.933215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:56.933215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:56.933215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:54:56.933215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:54:56.933215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:56.934073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:56.934923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:54:56.934923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:54:56.934923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:54:56.934923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:54:56.934923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:54:56.934923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:54:56.934923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:54:56.934923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:56.934923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 265/530] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:56.935381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25c899-ab4e-4175-b301-70dca056d844 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:56.989429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2c1a1-04cc-4dea-ae35-f0042740a14d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:56.990855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2c1a1-04cc-4dea-ae35-f0042740a14d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:57.038488 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25c899-ab4e-4175-b301-70dca056d844 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:57.061310 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:57.074199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4be08-31c8-4e02-89c7-a33494b801ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:57.074817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4be08-31c8-4e02-89c7-a33494b801ca None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:57.075093 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4be08-31c8-4e02-89c7-a33494b801ca None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:57.075362 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4be08-31c8-4e02-89c7-a33494b801ca None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:57.076098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4be08-31c8-4e02-89c7-a33494b801ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:57.076399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4be08-31c8-4e02-89c7-a33494b801ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:57.077834 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4be08-31c8-4e02-89c7-a33494b801ca None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:57.095906 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 266/531] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:54:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:57.096844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4be08-31c8-4e02-89c7-a33494b801ca None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:57.107283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:57.116643 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 266/532] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:54:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:54:57.117353 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2c1a1-04cc-4dea-ae35-f0042740a14d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:57.206478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4be08-31c8-4e02-89c7-a33494b801ca None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:57.237307 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2c1a1-04cc-4dea-ae35-f0042740a14d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:57.240079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4437a-a1a9-4e4f-9961-df2e3ef0b62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:57.240709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4437a-a1a9-4e4f-9961-df2e3ef0b62b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:57.240987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4437a-a1a9-4e4f-9961-df2e3ef0b62b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:57.241258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4437a-a1a9-4e4f-9961-df2e3ef0b62b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:57.241990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4437a-a1a9-4e4f-9961-df2e3ef0b62b None admin] RBAC: Adding query filter params (role.id=b256252b45e541a491376a134f502183, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=6e9209e77ce44b128e25a8ddcf78f8b5) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:57.242297 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4437a-a1a9-4e4f-9961-df2e3ef0b62b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:57.243761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4437a-a1a9-4e4f-9961-df2e3ef0b62b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:57.249382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 267/533] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:57 2022] GET /identity/v3/role_assignments?user.id=6e9209e77ce44b128e25a8ddcf78f8b5&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=b256252b45e541a491376a134f502183 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:57.249982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4437a-a1a9-4e4f-9961-df2e3ef0b62b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:57.284678 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e754078-fee9-49e5-920b-16abfcfd805c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:57.285855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e754078-fee9-49e5-920b-16abfcfd805c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:57.286175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e754078-fee9-49e5-920b-16abfcfd805c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:57.286442 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e754078-fee9-49e5-920b-16abfcfd805c None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:57.287160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e754078-fee9-49e5-920b-16abfcfd805c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:57.287473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e754078-fee9-49e5-920b-16abfcfd805c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:57.289135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e754078-fee9-49e5-920b-16abfcfd805c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:57.308087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:57.308289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 267/534] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:54:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:57.309494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e754078-fee9-49e5-920b-16abfcfd805c None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:57.370011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4437a-a1a9-4e4f-9961-df2e3ef0b62b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:57.376508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:57.409655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e7457-7323-40c9-80c3-e58a46576697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:57.410600 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e7457-7323-40c9-80c3-e58a46576697 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:57.411053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e7457-7323-40c9-80c3-e58a46576697 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:57.411566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e7457-7323-40c9-80c3-e58a46576697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:57.412651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e7457-7323-40c9-80c3-e58a46576697 None admin] RBAC: Adding query filter params (role.id=b256252b45e541a491376a134f502183, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=89df988d73ff48f3b7a92d335df9b633) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:57.413398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e7457-7323-40c9-80c3-e58a46576697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:57.415036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e7457-7323-40c9-80c3-e58a46576697 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:57.423247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 268/535] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:57 2022] GET /identity/v3/role_assignments?user.id=89df988d73ff48f3b7a92d335df9b633&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=b256252b45e541a491376a134f502183 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:57.424827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e7457-7323-40c9-80c3-e58a46576697 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:57.429675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e754078-fee9-49e5-920b-16abfcfd805c None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:57.476080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76283e-3816-464b-87c4-3f00e680d126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:57.476360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76283e-3816-464b-87c4-3f00e680d126 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:57.476723 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76283e-3816-464b-87c4-3f00e680d126 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:57.477084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76283e-3816-464b-87c4-3f00e680d126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:57.489825 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76283e-3816-464b-87c4-3f00e680d126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:57.492129 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76283e-3816-464b-87c4-3f00e680d126 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-ee76283e-3816-464b-87c4-3f00e680d126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:54:57.529927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:57.531366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:54:57.532638 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:54:57.532638 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:54:57.532638 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:57.532638 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 268/536] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:57.538406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76283e-3816-464b-87c4-3f00e680d126 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:57.562715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e7457-7323-40c9-80c3-e58a46576697 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:57.586569 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:57.650250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7a985-326b-4b45-b29f-1ac8e7762de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:57.651393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7a985-326b-4b45-b29f-1ac8e7762de7 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:57.651691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7a985-326b-4b45-b29f-1ac8e7762de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:57.651957 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7a985-326b-4b45-b29f-1ac8e7762de7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:57.652695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7a985-326b-4b45-b29f-1ac8e7762de7 None admin] RBAC: Adding query filter params (role.id=b256252b45e541a491376a134f502183, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=f24ee16c043244938901c4e850dea967) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:57.652995 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7a985-326b-4b45-b29f-1ac8e7762de7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:57.655472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7a985-326b-4b45-b29f-1ac8e7762de7 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:57.671798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 269/537] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:57 2022] GET /identity/v3/role_assignments?user.id=f24ee16c043244938901c4e850dea967&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=b256252b45e541a491376a134f502183 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:57.720629 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76283e-3816-464b-87c4-3f00e680d126 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:57.732889 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:57.761685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f78c8-33c2-4d3b-a378-50bd821d0ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:57.762230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f78c8-33c2-4d3b-a378-50bd821d0ced None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:57.762573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f78c8-33c2-4d3b-a378-50bd821d0ced None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:57.762842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f78c8-33c2-4d3b-a378-50bd821d0ced None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:57.763538 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f78c8-33c2-4d3b-a378-50bd821d0ced None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:57.763902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f78c8-33c2-4d3b-a378-50bd821d0ced None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:57.765413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f78c8-33c2-4d3b-a378-50bd821d0ced None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:57.779944 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 269/538] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:57.788778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7a985-326b-4b45-b29f-1ac8e7762de7 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:57.910423 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7a985-326b-4b45-b29f-1ac8e7762de7 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:57.944588 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b3b4b-69fe-437a-a6b7-bd99c285af51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:57.945228 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b3b4b-69fe-437a-a6b7-bd99c285af51 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:57.945499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b3b4b-69fe-437a-a6b7-bd99c285af51 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:57.945768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b3b4b-69fe-437a-a6b7-bd99c285af51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:57.952894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b3b4b-69fe-437a-a6b7-bd99c285af51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:57.954346 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b3b4b-69fe-437a-a6b7-bd99c285af51 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:57.966512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-d61b3b4b-69fe-437a-a6b7-bd99c285af51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:54:57.966512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:57.966512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:57.966512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:57.966512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:57.966512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:57.966512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:57.966512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:57.966512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:57.966512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:57.966512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:57.966512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:57.966512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:54:57.966512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:54:57.966512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:57.967404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:57.968547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:54:57.968547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:54:57.968547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:54:57.968547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:54:57.968547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:54:57.968547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:54:57.968547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:54:57.968547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:54:57.968547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 270/539] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:54:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:57.976578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f78c8-33c2-4d3b-a378-50bd821d0ced None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:58.118276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f78c8-33c2-4d3b-a378-50bd821d0ced None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:58.165695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bde371-7e0f-4ae6-bdac-42351fb51d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:58.166073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bde371-7e0f-4ae6-bdac-42351fb51d51 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:58.166349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bde371-7e0f-4ae6-bdac-42351fb51d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:58.166621 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bde371-7e0f-4ae6-bdac-42351fb51d51 None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:58.167320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bde371-7e0f-4ae6-bdac-42351fb51d51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:58.167644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bde371-7e0f-4ae6-bdac-42351fb51d51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:58.170721 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bde371-7e0f-4ae6-bdac-42351fb51d51 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:58.188832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 270/540] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:54:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:58.196628 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b3b4b-69fe-437a-a6b7-bd99c285af51 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:58.271883 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:58.359947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b3b4b-69fe-437a-a6b7-bd99c285af51 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:58.401285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4062d-f432-4f9c-b16d-cff8d722ab1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:58.401815 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4062d-f432-4f9c-b16d-cff8d722ab1e None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:58.402089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4062d-f432-4f9c-b16d-cff8d722ab1e None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:58.402357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4062d-f432-4f9c-b16d-cff8d722ab1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:58.403087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4062d-f432-4f9c-b16d-cff8d722ab1e None admin] RBAC: Adding query filter params (role.id=b256252b45e541a491376a134f502183, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=3aae65f9ec394f97aeafad43932dc49c) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:58.403388 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4062d-f432-4f9c-b16d-cff8d722ab1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:58.404840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4062d-f432-4f9c-b16d-cff8d722ab1e None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:58.410960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 271/541] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:54:58 2022] GET /identity/v3/role_assignments?user.id=3aae65f9ec394f97aeafad43932dc49c&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=b256252b45e541a491376a134f502183 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:58.911809 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09205e31-38b9-424f-83d5-20f673579a92 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:58.912381 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09205e31-38b9-424f-83d5-20f673579a92 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:58.912381 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09205e31-38b9-424f-83d5-20f673579a92 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:58.914071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 271/542] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:58.920284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2cfb2-f130-411c-8874-49566f55fb36 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:58.920809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2cfb2-f130-411c-8874-49566f55fb36 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:58.920809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2cfb2-f130-411c-8874-49566f55fb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:58.948506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-79d2cfb2-f130-411c-8874-49566f55fb36 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:54:58.949742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-79d2cfb2-f130-411c-8874-49566f55fb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:54:59.046055 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 272/543] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:54:59.402017 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09205e31-38b9-424f-83d5-20f673579a92 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:59.438890 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:59.541064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09205e31-38b9-424f-83d5-20f673579a92 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:59.582517 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927c73c-d406-4e2e-8aa8-60fab788f55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:59.583555 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927c73c-d406-4e2e-8aa8-60fab788f55f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:59.584029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927c73c-d406-4e2e-8aa8-60fab788f55f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:59.584530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927c73c-d406-4e2e-8aa8-60fab788f55f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:59.600827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927c73c-d406-4e2e-8aa8-60fab788f55f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-f927c73c-d406-4e2e-8aa8-60fab788f55f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:54:59.606145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:54:59.607039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:54:59.607877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:54:59.607877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:54:59.607877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:54:59.607877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:54:59.607877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:54:59.607877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:54:59.607877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:54:59.607877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:54:59.607877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:54:59.607877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:54:59.607877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:54:59.607877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:54:59.610144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 272/544] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:54:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:54:59.616210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2cfb2-f130-411c-8874-49566f55fb36 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:59.659826 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:59.737701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2cfb2-f130-411c-8874-49566f55fb36 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:59.772098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17e532-b80f-42d8-8128-4cda49e5491e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:59.772714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17e532-b80f-42d8-8128-4cda49e5491e None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:59.772992 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17e532-b80f-42d8-8128-4cda49e5491e None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:59.773262 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17e532-b80f-42d8-8128-4cda49e5491e None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:59.773940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17e532-b80f-42d8-8128-4cda49e5491e None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:54:59.774240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17e532-b80f-42d8-8128-4cda49e5491e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:59.775638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17e532-b80f-42d8-8128-4cda49e5491e None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:54:59.784758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 273/545] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:54:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:54:59.792837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927c73c-d406-4e2e-8aa8-60fab788f55f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:54:59.931469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:54:59.942041 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927c73c-d406-4e2e-8aa8-60fab788f55f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:54:59.963961 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f43df-2016-45f9-93a9-51414b143636 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:59.964474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f43df-2016-45f9-93a9-51414b143636 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:59.964474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f43df-2016-45f9-93a9-51414b143636 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:59.966102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 274/546] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:54:59.976856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcfa6d-0bbb-4234-93c2-63679fe6aa9c None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:59.977133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcfa6d-0bbb-4234-93c2-63679fe6aa9c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:59.977408 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcfa6d-0bbb-4234-93c2-63679fe6aa9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:59.983382 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522582eb-2a61-4eb5-9d2e-17f51ace6102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:54:59.984051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522582eb-2a61-4eb5-9d2e-17f51ace6102 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:54:59.984324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522582eb-2a61-4eb5-9d2e-17f51ace6102 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:54:59.984593 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522582eb-2a61-4eb5-9d2e-17f51ace6102 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:54:59.995915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522582eb-2a61-4eb5-9d2e-17f51ace6102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:54:59.997317 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522582eb-2a61-4eb5-9d2e-17f51ace6102 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-522582eb-2a61-4eb5-9d2e-17f51ace6102 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:00.012016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:00.012977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:00.014523 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:00.014523 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:00.014523 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:00.014523 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 273/547] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:00.014787 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-11fcfa6d-0bbb-4234-93c2-63679fe6aa9c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:00.015800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-11fcfa6d-0bbb-4234-93c2-63679fe6aa9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:00.024872 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522582eb-2a61-4eb5-9d2e-17f51ace6102 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:00.125166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 275/548] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:54:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:55:00.176750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522582eb-2a61-4eb5-9d2e-17f51ace6102 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:00.221724 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77878665-0bcf-4d09-becc-306f4a3cdc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:00.222345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77878665-0bcf-4d09-becc-306f4a3cdc61 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:00.222623 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77878665-0bcf-4d09-becc-306f4a3cdc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:00.222893 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77878665-0bcf-4d09-becc-306f4a3cdc61 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:00.223570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77878665-0bcf-4d09-becc-306f4a3cdc61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:00.223874 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77878665-0bcf-4d09-becc-306f4a3cdc61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:00.225267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77878665-0bcf-4d09-becc-306f4a3cdc61 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:00.242716 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 274/549] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:00.247961 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcfa6d-0bbb-4234-93c2-63679fe6aa9c None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:00.277202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:00.294793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6ee96-4f59-49bc-8938-5b8d43a82089 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:00.295098 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6ee96-4f59-49bc-8938-5b8d43a82089 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:00.295337 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a6ee96-4f59-49bc-8938-5b8d43a82089 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:00.296715 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 275/550] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:00.317386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d66bf-5c36-40c9-847c-d343773ffe17 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:00.317681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d66bf-5c36-40c9-847c-d343773ffe17 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:00.317992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d66bf-5c36-40c9-847c-d343773ffe17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:00.329178 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:00.343024 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-e94d66bf-5c36-40c9-847c-d343773ffe17 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:00.348421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e94d66bf-5c36-40c9-847c-d343773ffe17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:00.400222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcfa6d-0bbb-4234-93c2-63679fe6aa9c None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:00.465560 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dbaab-e3e9-4670-9ef6-448be1b73ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:00.466213 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dbaab-e3e9-4670-9ef6-448be1b73ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:00.466489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dbaab-e3e9-4670-9ef6-448be1b73ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:00.466718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 276/551] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:55:00.466833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dbaab-e3e9-4670-9ef6-448be1b73ff8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:00.467612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d66bf-5c36-40c9-847c-d343773ffe17 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:00.474474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47dbaab-e3e9-4670-9ef6-448be1b73ff8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:00.476057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47dbaab-e3e9-4670-9ef6-448be1b73ff8 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-f47dbaab-e3e9-4670-9ef6-448be1b73ff8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:00.488699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:00.489859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:00.491127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:00.491127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:00.491127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:00.491127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:00.491127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:00.491127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:00.491127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:00.491127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:00.491127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 15:55:00.491127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:00.491127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 276/552] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:55:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:00.495404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dbaab-e3e9-4670-9ef6-448be1b73ff8 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:00.662740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d66bf-5c36-40c9-847c-d343773ffe17 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:00.663248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:00.691761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dbaab-e3e9-4670-9ef6-448be1b73ff8 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:00.703857 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:00.709167 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c60266-9e63-430b-a685-b26df5c1f59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:00.710112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c60266-9e63-430b-a685-b26df5c1f59f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:00.710589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c60266-9e63-430b-a685-b26df5c1f59f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:00.711122 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c60266-9e63-430b-a685-b26df5c1f59f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:00.722920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c60266-9e63-430b-a685-b26df5c1f59f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-d2c60266-9e63-430b-a685-b26df5c1f59f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:00.728619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:00.729722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:00.730557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:00.730557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:00.730557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:00.730557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:00.730557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:00.730557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:00.730557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:00.730557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:00.730557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:00.730557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:00.730557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:00.730557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:00.737513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 277/553] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:55:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:00.737513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b569c7-be96-44e8-9ced-15938fdf20f4 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:00.737513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b569c7-be96-44e8-9ced-15938fdf20f4 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:00.737513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b569c7-be96-44e8-9ced-15938fdf20f4 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:00.738004 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e26ee6-c888-4a97-ac2a-60a3755197e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:00.738624 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e26ee6-c888-4a97-ac2a-60a3755197e6 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:00.738892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e26ee6-c888-4a97-ac2a-60a3755197e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:00.739152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e26ee6-c888-4a97-ac2a-60a3755197e6 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:00.739684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 278/554] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:00.740334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e26ee6-c888-4a97-ac2a-60a3755197e6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:00.740638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e26ee6-c888-4a97-ac2a-60a3755197e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:00.741061 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b569c7-be96-44e8-9ced-15938fdf20f4 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:00.745892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e26ee6-c888-4a97-ac2a-60a3755197e6 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:00.764521 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 277/555] 10.222.0.27 () {56 vars in 1135 bytes} [Thu Sep 8 15:55:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:00.766694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c90a1-ab1c-4f8c-9540-ec5e1404f93c None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:00.767178 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c90a1-ab1c-4f8c-9540-ec5e1404f93c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:00.767671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c90a1-ab1c-4f8c-9540-ec5e1404f93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:00.790751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-249c90a1-ab1c-4f8c-9540-ec5e1404f93c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:00.792786 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-249c90a1-ab1c-4f8c-9540-ec5e1404f93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:00.858955 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b569c7-be96-44e8-9ced-15938fdf20f4 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:00.864454 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 278/556] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:55:00.865135 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c90a1-ab1c-4f8c-9540-ec5e1404f93c None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:00.916898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14005356-4a95-4bb7-8468-dcb3d0375a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:00.918353 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14005356-4a95-4bb7-8468-dcb3d0375a73 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:00.918832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14005356-4a95-4bb7-8468-dcb3d0375a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:00.919591 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14005356-4a95-4bb7-8468-dcb3d0375a73 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:00.920719 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14005356-4a95-4bb7-8468-dcb3d0375a73 None admin] RBAC: Adding query filter params (name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:00.921622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14005356-4a95-4bb7-8468-dcb3d0375a73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:00.923755 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14005356-4a95-4bb7-8468-dcb3d0375a73 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:00.932239 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:00.935161 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 279/557] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:55:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:00.936636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14005356-4a95-4bb7-8468-dcb3d0375a73 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:00.968976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c90a1-ab1c-4f8c-9540-ec5e1404f93c None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:00.995084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:01.006405 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af3e45-8408-455a-9119-b1411f09b439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:01.007173 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af3e45-8408-455a-9119-b1411f09b439 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:01.007474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af3e45-8408-455a-9119-b1411f09b439 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:01.008214 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7af3e45-8408-455a-9119-b1411f09b439 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:01.044280 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14005356-4a95-4bb7-8468-dcb3d0375a73 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:01.077782 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b2000-0b46-4ffa-8c4c-3fc35a885050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:01.078415 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b2000-0b46-4ffa-8c4c-3fc35a885050 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:01.078688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b2000-0b46-4ffa-8c4c-3fc35a885050 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:01.078955 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af3e45-8408-455a-9119-b1411f09b439 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:01.079070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b2000-0b46-4ffa-8c4c-3fc35a885050 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:01.080432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7af3e45-8408-455a-9119-b1411f09b439 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:01.089618 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b2000-0b46-4ffa-8c4c-3fc35a885050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-aa2b2000-0b46-4ffa-8c4c-3fc35a885050 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:01.094850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:01.095992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:01.096949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:01.096949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:01.096949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:01.096949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:01.096949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:01.096949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:01.096949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:01.096949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:01.096949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:01.096949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:01.096949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:01.096949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:01.096949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 280/558] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:55:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:01.098100 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85f8f1-015c-4243-bda7-0329b48bbdf7 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:01.098100 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85f8f1-015c-4243-bda7-0329b48bbdf7 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:01.098100 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85f8f1-015c-4243-bda7-0329b48bbdf7 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:01.103605 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 281/559] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:01.103605 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85f8f1-015c-4243-bda7-0329b48bbdf7 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:01.156221 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:01.192201 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 279/560] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:55:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:55:01.193060 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af3e45-8408-455a-9119-b1411f09b439 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:01.202837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85f8f1-015c-4243-bda7-0329b48bbdf7 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:01.216753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:01.236586 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e7872-0f30-4549-870b-d5d5e3074976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:01.237239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e7872-0f30-4549-870b-d5d5e3074976 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:01.237510 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e7872-0f30-4549-870b-d5d5e3074976 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:01.237774 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e7872-0f30-4549-870b-d5d5e3074976 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:01.248216 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e7872-0f30-4549-870b-d5d5e3074976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:01.249678 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89e7872-0f30-4549-870b-d5d5e3074976 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-a89e7872-0f30-4549-870b-d5d5e3074976 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:01.264390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:01.265407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:01.266289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:01.266289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:01.266289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:01.266289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 282/561] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:01.266682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54577c-982a-4a39-b376-735b79dc16e4 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:01.266986 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54577c-982a-4a39-b376-735b79dc16e4 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:01.267211 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e54577c-982a-4a39-b376-735b79dc16e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:01.284676 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-4e54577c-982a-4a39-b376-735b79dc16e4 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:01.285862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-4e54577c-982a-4a39-b376-735b79dc16e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:01.295305 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7af3e45-8408-455a-9119-b1411f09b439 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:01.330490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30dc9d-8170-4ad8-91ee-0748201b24f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:01.331122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30dc9d-8170-4ad8-91ee-0748201b24f8 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:01.331397 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30dc9d-8170-4ad8-91ee-0748201b24f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:01.331696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30dc9d-8170-4ad8-91ee-0748201b24f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:01.332362 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30dc9d-8170-4ad8-91ee-0748201b24f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:01.332659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30dc9d-8170-4ad8-91ee-0748201b24f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:01.334015 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30dc9d-8170-4ad8-91ee-0748201b24f8 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:01.342247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 280/562] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:55:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:01.343028 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30dc9d-8170-4ad8-91ee-0748201b24f8 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:01.354854 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 283/563] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:55:01.355720 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54577c-982a-4a39-b376-735b79dc16e4 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:01.376443 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:01.447651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30dc9d-8170-4ad8-91ee-0748201b24f8 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:01.464401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:01.471033 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e54577c-982a-4a39-b376-735b79dc16e4 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:01.485209 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8cb19-be7d-4e0e-8c11-a0e3e9cc9b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:01.485838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8cb19-be7d-4e0e-8c11-a0e3e9cc9b90 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:01.486112 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8cb19-be7d-4e0e-8c11-a0e3e9cc9b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:01.486387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e8cb19-be7d-4e0e-8c11-a0e3e9cc9b90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:01.498179 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e8cb19-be7d-4e0e-8c11-a0e3e9cc9b90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-f1e8cb19-be7d-4e0e-8c11-a0e3e9cc9b90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:01.505596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:01.506674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:01.507573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:01.507573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:01.507573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:01.507573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:01.507573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:01.507573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:01.507573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:01.507573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:01.507573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:01.507573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:01.507573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:01.507573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:01.507573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 281/564] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:55:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:01.507573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8cb19-be7d-4e0e-8c11-a0e3e9cc9b90 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:01.512749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f919ad-1542-4f16-a7ca-9b3a59dd5cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:01.513395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f919ad-1542-4f16-a7ca-9b3a59dd5cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:01.513689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f919ad-1542-4f16-a7ca-9b3a59dd5cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:01.513922 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f919ad-1542-4f16-a7ca-9b3a59dd5cc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:01.524457 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f919ad-1542-4f16-a7ca-9b3a59dd5cc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:01.525949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f919ad-1542-4f16-a7ca-9b3a59dd5cc4 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-35f919ad-1542-4f16-a7ca-9b3a59dd5cc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:01.541054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:01.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:01.543333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:01.543333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:01.543333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:01.543333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 284/565] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:01.543333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f919ad-1542-4f16-a7ca-9b3a59dd5cc4 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:01.606324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:01.617413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e8cb19-be7d-4e0e-8c11-a0e3e9cc9b90 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:01.650590 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f919ad-1542-4f16-a7ca-9b3a59dd5cc4 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:01.652463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6690e-7a81-426d-8aa3-ee812c462dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:01.653078 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6690e-7a81-426d-8aa3-ee812c462dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:01.653352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6690e-7a81-426d-8aa3-ee812c462dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:01.653619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6690e-7a81-426d-8aa3-ee812c462dc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:01.654263 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6690e-7a81-426d-8aa3-ee812c462dc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:01.654587 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6690e-7a81-426d-8aa3-ee812c462dc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:01.655973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b6690e-7a81-426d-8aa3-ee812c462dc2 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:01.667874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 282/566] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:01.668676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6690e-7a81-426d-8aa3-ee812c462dc2 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:01.684853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1ded7-4926-4fcb-a07e-937a10965f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:01.685545 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ded7-4926-4fcb-a07e-937a10965f7f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:01.685821 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ded7-4926-4fcb-a07e-937a10965f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:01.686110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1ded7-4926-4fcb-a07e-937a10965f7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:01.696814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1ded7-4926-4fcb-a07e-937a10965f7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:01.698338 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1ded7-4926-4fcb-a07e-937a10965f7f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:01.706441 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-33a1ded7-4926-4fcb-a07e-937a10965f7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:01.713627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:01.714569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:01.715492 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:01.715492 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:01.715492 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:01.715492 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 285/567] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:01.715913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1ded7-4926-4fcb-a07e-937a10965f7f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:01.769437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b6690e-7a81-426d-8aa3-ee812c462dc2 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:01.803134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3f65f-be53-45bc-8ea2-9011599a47d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:01.803790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3f65f-be53-45bc-8ea2-9011599a47d5 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:01.804069 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3f65f-be53-45bc-8ea2-9011599a47d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:01.804336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3f65f-be53-45bc-8ea2-9011599a47d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:01.805007 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3f65f-be53-45bc-8ea2-9011599a47d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:01.805326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3f65f-be53-45bc-8ea2-9011599a47d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:01.806701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff3f65f-be53-45bc-8ea2-9011599a47d5 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:01.814468 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 283/568] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:55:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:01.815315 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3f65f-be53-45bc-8ea2-9011599a47d5 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:01.815788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1ded7-4926-4fcb-a07e-937a10965f7f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:01.829888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:01.850114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4ffa9-fe51-4315-9d8b-4581838309ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:01.850729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4ffa9-fe51-4315-9d8b-4581838309ce None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:01.851006 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4ffa9-fe51-4315-9d8b-4581838309ce None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:01.851278 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4ffa9-fe51-4315-9d8b-4581838309ce None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:01.851959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4ffa9-fe51-4315-9d8b-4581838309ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:01.852261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4ffa9-fe51-4315-9d8b-4581838309ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:01.853641 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4ffa9-fe51-4315-9d8b-4581838309ce None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:01.865216 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 286/569] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:01.866075 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4ffa9-fe51-4315-9d8b-4581838309ce None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:01.911954 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff3f65f-be53-45bc-8ea2-9011599a47d5 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:01.931672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:01.947857 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929add8-bd54-4bc2-b931-6e28dcc65b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:01.948569 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929add8-bd54-4bc2-b931-6e28dcc65b1d None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:01.948777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929add8-bd54-4bc2-b931-6e28dcc65b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:01.949047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929add8-bd54-4bc2-b931-6e28dcc65b1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:01.961042 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929add8-bd54-4bc2-b931-6e28dcc65b1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:01.967775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4ffa9-fe51-4315-9d8b-4581838309ce None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-4929add8-bd54-4bc2-b931-6e28dcc65b1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:01.968238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:01.969671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:01.970655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:01.970655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:01.970655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:01.970655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:01.970655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:01.970655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:01.970655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:01.970655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:01.970655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:01.970655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:01.970655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:01.970655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:01.970655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 284/570] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:55:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:01.970655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929add8-bd54-4bc2-b931-6e28dcc65b1d None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:02.005160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d178b-5db2-48a4-a976-e01dade253d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:02.005822 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d178b-5db2-48a4-a976-e01dade253d7 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:02.006101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d178b-5db2-48a4-a976-e01dade253d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:02.006369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d178b-5db2-48a4-a976-e01dade253d7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:02.013323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d178b-5db2-48a4-a976-e01dade253d7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:02.014776 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d178b-5db2-48a4-a976-e01dade253d7 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-461d178b-5db2-48a4-a976-e01dade253d7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:02.026128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:02.027070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:02.028051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:02.028051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:02.028051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:02.028051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:02.028051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:02.028051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:02.028051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:02.028051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:02.028051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 15:55:02.028051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:02.028051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 287/571] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:55:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:02.028051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d178b-5db2-48a4-a976-e01dade253d7 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:02.043162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:02.072524 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929add8-bd54-4bc2-b931-6e28dcc65b1d None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:02.105519 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4591d-b98a-4e86-a9c7-1cd50c236855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:02.106136 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4591d-b98a-4e86-a9c7-1cd50c236855 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:02.106416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4591d-b98a-4e86-a9c7-1cd50c236855 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:02.106686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f4591d-b98a-4e86-a9c7-1cd50c236855 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:02.107327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4591d-b98a-4e86-a9c7-1cd50c236855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:02.107662 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4591d-b98a-4e86-a9c7-1cd50c236855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:02.109043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f4591d-b98a-4e86-a9c7-1cd50c236855 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:02.120413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 285/572] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:02.121203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4591d-b98a-4e86-a9c7-1cd50c236855 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:02.133249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d178b-5db2-48a4-a976-e01dade253d7 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:02.153049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:02.168554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239ab8a-6ccc-4789-95b3-ee2826ea9261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:02.169199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239ab8a-6ccc-4789-95b3-ee2826ea9261 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:02.169473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239ab8a-6ccc-4789-95b3-ee2826ea9261 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:02.169739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3239ab8a-6ccc-4789-95b3-ee2826ea9261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:02.180829 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3239ab8a-6ccc-4789-95b3-ee2826ea9261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:02.183057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3239ab8a-6ccc-4789-95b3-ee2826ea9261 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-3239ab8a-6ccc-4789-95b3-ee2826ea9261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:02.197219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:02.198360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:02.199915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:02.199915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:02.199915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:02.199915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 288/573] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:02.201071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239ab8a-6ccc-4789-95b3-ee2826ea9261 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:02.219671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f4591d-b98a-4e86-a9c7-1cd50c236855 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:02.248581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:02.253679 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c17e2-be2b-4554-ac3f-49999eeb8a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:02.254311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c17e2-be2b-4554-ac3f-49999eeb8a95 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:02.254586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c17e2-be2b-4554-ac3f-49999eeb8a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:02.254850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c17e2-be2b-4554-ac3f-49999eeb8a95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:02.261505 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c17e2-be2b-4554-ac3f-49999eeb8a95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:02.262970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c17e2-be2b-4554-ac3f-49999eeb8a95 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:02.275037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-d69c17e2-be2b-4554-ac3f-49999eeb8a95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:02.275037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:02.275037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:02.275037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:02.275037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:02.275037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:02.275037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:02.275037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:02.275037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:02.275037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:02.275037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:02.275037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:02.275037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:02.275037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:02.275037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:02.276163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:02.277061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:02.277061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:02.277061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:02.277061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:02.277061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:02.277061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:02.277061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:02.277061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:02.277061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 286/574] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:02.277061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c17e2-be2b-4554-ac3f-49999eeb8a95 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:02.304814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3239ab8a-6ccc-4789-95b3-ee2826ea9261 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:02.339316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79f425-bdd3-4ff4-a7f0-c38cd1cd6f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:02.339936 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79f425-bdd3-4ff4-a7f0-c38cd1cd6f5f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:02.340203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79f425-bdd3-4ff4-a7f0-c38cd1cd6f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:02.340469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79f425-bdd3-4ff4-a7f0-c38cd1cd6f5f None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:02.341135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79f425-bdd3-4ff4-a7f0-c38cd1cd6f5f None admin] RBAC: Adding query filter params (name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:02.341432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79f425-bdd3-4ff4-a7f0-c38cd1cd6f5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:02.342871 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79f425-bdd3-4ff4-a7f0-c38cd1cd6f5f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:02.350561 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 289/575] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:55:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:02.351340 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79f425-bdd3-4ff4-a7f0-c38cd1cd6f5f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:02.384302 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c17e2-be2b-4554-ac3f-49999eeb8a95 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:02.385030 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:02.418470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5879ae-fd64-450a-aae4-a244ac5c3903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:02.419193 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5879ae-fd64-450a-aae4-a244ac5c3903 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:02.419516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5879ae-fd64-450a-aae4-a244ac5c3903 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:02.419789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5879ae-fd64-450a-aae4-a244ac5c3903 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:02.420536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5879ae-fd64-450a-aae4-a244ac5c3903 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:02.420843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5879ae-fd64-450a-aae4-a244ac5c3903 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:02.422307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5879ae-fd64-450a-aae4-a244ac5c3903 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:02.439714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 287/576] 10.222.0.27 () {56 vars in 1137 bytes} [Thu Sep 8 15:55:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:02.440532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5879ae-fd64-450a-aae4-a244ac5c3903 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:02.453556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79f425-bdd3-4ff4-a7f0-c38cd1cd6f5f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:02.486944 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd269dc-daa2-4849-ac9b-ab1af6721349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:02.487606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd269dc-daa2-4849-ac9b-ab1af6721349 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:02.487880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd269dc-daa2-4849-ac9b-ab1af6721349 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:02.488143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd269dc-daa2-4849-ac9b-ab1af6721349 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:02.489874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:02.496004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd269dc-daa2-4849-ac9b-ab1af6721349 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:02.497450 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd269dc-daa2-4849-ac9b-ab1af6721349 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-ccd269dc-daa2-4849-ac9b-ab1af6721349 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:02.508304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:02.509345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:02.510314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:02.510314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:02.510314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:02.510314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:02.510314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:02.510314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:02.510314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:02.510314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:02.510314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:55:02.510314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:02.514005 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 290/577] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Sep 8 15:55:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:02.514850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd269dc-daa2-4849-ac9b-ab1af6721349 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:02.537747 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5879ae-fd64-450a-aae4-a244ac5c3903 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:02.572283 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef07e26-41d8-440e-b71d-a50a052c0874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:02.572894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef07e26-41d8-440e-b71d-a50a052c0874 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:02.573189 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef07e26-41d8-440e-b71d-a50a052c0874 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:02.573465 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef07e26-41d8-440e-b71d-a50a052c0874 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:02.574103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef07e26-41d8-440e-b71d-a50a052c0874 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:02.574399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef07e26-41d8-440e-b71d-a50a052c0874 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:02.575799 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef07e26-41d8-440e-b71d-a50a052c0874 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:02.586822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 288/578] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:02.588000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef07e26-41d8-440e-b71d-a50a052c0874 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:02.609561 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:02.617376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd269dc-daa2-4849-ac9b-ab1af6721349 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:02.658816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617b9ce-459a-43d4-beac-5f956021a837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:02.659436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617b9ce-459a-43d4-beac-5f956021a837 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:02.659739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617b9ce-459a-43d4-beac-5f956021a837 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:02.660083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617b9ce-459a-43d4-beac-5f956021a837 None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:02.660705 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617b9ce-459a-43d4-beac-5f956021a837 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:02.661006 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617b9ce-459a-43d4-beac-5f956021a837 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:02.662451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3617b9ce-459a-43d4-beac-5f956021a837 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:02.674343 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 291/579] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:02.675191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617b9ce-459a-43d4-beac-5f956021a837 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:02.692331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef07e26-41d8-440e-b71d-a50a052c0874 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:02.702739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:02.726056 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f077b-b537-429f-be04-597bd8975456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:02.726618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f077b-b537-429f-be04-597bd8975456 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:02.726885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f077b-b537-429f-be04-597bd8975456 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:02.727147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f077b-b537-429f-be04-597bd8975456 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:02.737418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f077b-b537-429f-be04-597bd8975456 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:02.738887 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f077b-b537-429f-be04-597bd8975456 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-8d4f077b-b537-429f-be04-597bd8975456 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:02.752498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:02.753508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:02.754483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:02.754483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:02.754483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:02.754483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 289/580] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:02.754483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f077b-b537-429f-be04-597bd8975456 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:02.778150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617b9ce-459a-43d4-beac-5f956021a837 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:02.811113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896bce86-d5c9-44b9-a530-88d9d6b51c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:02.811771 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bce86-d5c9-44b9-a530-88d9d6b51c43 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:02.812044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bce86-d5c9-44b9-a530-88d9d6b51c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:02.812315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896bce86-d5c9-44b9-a530-88d9d6b51c43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:02.822670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896bce86-d5c9-44b9-a530-88d9d6b51c43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:02.824163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896bce86-d5c9-44b9-a530-88d9d6b51c43 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-896bce86-d5c9-44b9-a530-88d9d6b51c43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:02.839132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:02.840132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:02.841027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:02.841027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:02.841027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:02.841027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 292/581] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:02.841027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896bce86-d5c9-44b9-a530-88d9d6b51c43 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:02.849405 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:02.853624 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f077b-b537-429f-be04-597bd8975456 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:02.887260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78412f-9c19-414b-953c-271bf2283855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:02.887926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78412f-9c19-414b-953c-271bf2283855 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:02.888205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78412f-9c19-414b-953c-271bf2283855 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:02.888478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78412f-9c19-414b-953c-271bf2283855 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:02.889212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78412f-9c19-414b-953c-271bf2283855 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:02.889518 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78412f-9c19-414b-953c-271bf2283855 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:02.890957 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78412f-9c19-414b-953c-271bf2283855 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:02.908203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 290/582] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Sep 8 15:55:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:02.909090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78412f-9c19-414b-953c-271bf2283855 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:02.930492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:02.944073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896bce86-d5c9-44b9-a530-88d9d6b51c43 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:02.979209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeafa35-2e0f-4d25-ae5b-1ee59f6ba630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:02.979840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeafa35-2e0f-4d25-ae5b-1ee59f6ba630 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:02.980124 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeafa35-2e0f-4d25-ae5b-1ee59f6ba630 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:02.980397 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeafa35-2e0f-4d25-ae5b-1ee59f6ba630 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:02.987877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeafa35-2e0f-4d25-ae5b-1ee59f6ba630 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:02.989348 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeafa35-2e0f-4d25-ae5b-1ee59f6ba630 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-4eeafa35-2e0f-4d25-ae5b-1ee59f6ba630 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:03.001103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:03.002133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:03.003187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:03.003187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:03.003187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:03.003187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:03.003187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:03.003187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:03.003187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:03.003187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:03.003187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 15:55:03.003187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:03.003187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 293/583] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:55:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:03.003187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeafa35-2e0f-4d25-ae5b-1ee59f6ba630 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:03.011917 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78412f-9c19-414b-953c-271bf2283855 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:03.046133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fb7d3-db78-4062-bc2c-bc2885fcf03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:03.046755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fb7d3-db78-4062-bc2c-bc2885fcf03b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:03.047049 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fb7d3-db78-4062-bc2c-bc2885fcf03b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:03.047314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5fb7d3-db78-4062-bc2c-bc2885fcf03b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:03.048062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fb7d3-db78-4062-bc2c-bc2885fcf03b None admin] RBAC: Adding query filter params (role.id=b256252b45e541a491376a134f502183, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=de54687547a14ab89cc41a1fabaab400) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:03.048379 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fb7d3-db78-4062-bc2c-bc2885fcf03b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:03.049879 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5fb7d3-db78-4062-bc2c-bc2885fcf03b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:03.055166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 291/584] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:55:02 2022] GET /identity/v3/role_assignments?user.id=de54687547a14ab89cc41a1fabaab400&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=b256252b45e541a491376a134f502183 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:03.055976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fb7d3-db78-4062-bc2c-bc2885fcf03b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:03.068467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:03.110428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeafa35-2e0f-4d25-ae5b-1ee59f6ba630 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:03.149266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2887d87-929b-4146-b7f9-c9aad4418226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:03.149889 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2887d87-929b-4146-b7f9-c9aad4418226 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:03.150143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2887d87-929b-4146-b7f9-c9aad4418226 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:03.150419 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2887d87-929b-4146-b7f9-c9aad4418226 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:03.151079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2887d87-929b-4146-b7f9-c9aad4418226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:03.151366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2887d87-929b-4146-b7f9-c9aad4418226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:03.151717 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:03.152842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2887d87-929b-4146-b7f9-c9aad4418226 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:03.172758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5fb7d3-db78-4062-bc2c-bc2885fcf03b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:03.174004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 294/585] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:03.174886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2887d87-929b-4146-b7f9-c9aad4418226 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:03.214749 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028adff7-1bfb-4c67-87da-8bfe686af2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:03.215437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028adff7-1bfb-4c67-87da-8bfe686af2fc None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:03.215771 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028adff7-1bfb-4c67-87da-8bfe686af2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:03.216297 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028adff7-1bfb-4c67-87da-8bfe686af2fc None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:03.217910 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028adff7-1bfb-4c67-87da-8bfe686af2fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:03.218227 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028adff7-1bfb-4c67-87da-8bfe686af2fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:03.219788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028adff7-1bfb-4c67-87da-8bfe686af2fc None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:03.235568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 292/586] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:03.236030 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028adff7-1bfb-4c67-87da-8bfe686af2fc None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:03.287415 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2887d87-929b-4146-b7f9-c9aad4418226 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:03.313142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:03.322509 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8610082-0a43-4a97-93a4-c53af6ab13f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:03.323201 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8610082-0a43-4a97-93a4-c53af6ab13f5 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:03.323492 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8610082-0a43-4a97-93a4-c53af6ab13f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:03.323760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8610082-0a43-4a97-93a4-c53af6ab13f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:03.334836 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8610082-0a43-4a97-93a4-c53af6ab13f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:03.336310 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8610082-0a43-4a97-93a4-c53af6ab13f5 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:03.345498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028adff7-1bfb-4c67-87da-8bfe686af2fc None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-b8610082-0a43-4a97-93a4-c53af6ab13f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:03.351073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:03.352064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:03.352969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:03.352969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:03.352969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:03.356805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 295/587] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:03.357525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8610082-0a43-4a97-93a4-c53af6ab13f5 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:03.381571 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d142b76-8117-4696-bac2-aaf48d0f55f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:03.382180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d142b76-8117-4696-bac2-aaf48d0f55f0 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:03.382556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d142b76-8117-4696-bac2-aaf48d0f55f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:03.382818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d142b76-8117-4696-bac2-aaf48d0f55f0 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:03.383676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d142b76-8117-4696-bac2-aaf48d0f55f0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:03.383966 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d142b76-8117-4696-bac2-aaf48d0f55f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:03.385504 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d142b76-8117-4696-bac2-aaf48d0f55f0 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:03.405013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 293/588] 10.222.0.27 () {56 vars in 1137 bytes} [Thu Sep 8 15:55:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:03.406206 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d142b76-8117-4696-bac2-aaf48d0f55f0 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:03.409121 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:03.463420 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8610082-0a43-4a97-93a4-c53af6ab13f5 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:03.497714 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779357a7-c6a1-454f-ae00-15055a5e3113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:03.498265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779357a7-c6a1-454f-ae00-15055a5e3113 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:03.498542 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779357a7-c6a1-454f-ae00-15055a5e3113 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:03.498822 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779357a7-c6a1-454f-ae00-15055a5e3113 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:03.506764 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779357a7-c6a1-454f-ae00-15055a5e3113 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:03.508238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779357a7-c6a1-454f-ae00-15055a5e3113 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:03.521713 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d142b76-8117-4696-bac2-aaf48d0f55f0 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-779357a7-c6a1-454f-ae00-15055a5e3113 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:03.522314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:03.523227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:03.524276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:03.524276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:03.524276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:03.524276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:03.524276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:03.524276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:03.524276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:03.524276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:03.524276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 15:55:03.524276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:03.524276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 296/589] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:55:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:03.524276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779357a7-c6a1-454f-ae00-15055a5e3113 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:03.560921 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191b9ef-3fd4-4f6a-b975-de7d69a21499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:03.561573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191b9ef-3fd4-4f6a-b975-de7d69a21499 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:03.561831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191b9ef-3fd4-4f6a-b975-de7d69a21499 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:03.562098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191b9ef-3fd4-4f6a-b975-de7d69a21499 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:03.569754 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:03.571549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191b9ef-3fd4-4f6a-b975-de7d69a21499 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:03.573387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191b9ef-3fd4-4f6a-b975-de7d69a21499 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:03.584897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-f191b9ef-3fd4-4f6a-b975-de7d69a21499 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:03.584897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:03.584897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:03.584897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:03.584897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:03.584897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:03.584897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:03.584897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:03.584897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:03.584897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:03.584897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:03.584897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:03.584897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:03.584897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:03.584897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:03.585850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:03.586767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:03.586767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:03.586767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:03.586767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:03.586767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:03.586767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:03.586767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:03.586767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:03.586767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 294/590] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:03.586767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191b9ef-3fd4-4f6a-b975-de7d69a21499 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:03.638749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779357a7-c6a1-454f-ae00-15055a5e3113 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:03.656776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:03.679561 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd94b05-5eb7-4a07-9b37-3b723786ab11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:03.680177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd94b05-5eb7-4a07-9b37-3b723786ab11 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:03.680453 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd94b05-5eb7-4a07-9b37-3b723786ab11 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:03.680734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd94b05-5eb7-4a07-9b37-3b723786ab11 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:03.681386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd94b05-5eb7-4a07-9b37-3b723786ab11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:03.681689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd94b05-5eb7-4a07-9b37-3b723786ab11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:03.683096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd94b05-5eb7-4a07-9b37-3b723786ab11 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:03.695910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 297/591] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:03.696747 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd94b05-5eb7-4a07-9b37-3b723786ab11 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:03.712703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191b9ef-3fd4-4f6a-b975-de7d69a21499 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:03.750176 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc56f0-1edb-40dc-9986-cf2c05f93c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:03.750733 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc56f0-1edb-40dc-9986-cf2c05f93c9b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:03.751016 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc56f0-1edb-40dc-9986-cf2c05f93c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:03.751293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc56f0-1edb-40dc-9986-cf2c05f93c9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:03.762784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc56f0-1edb-40dc-9986-cf2c05f93c9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:03.764307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cc56f0-1edb-40dc-9986-cf2c05f93c9b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-a8cc56f0-1edb-40dc-9986-cf2c05f93c9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:03.779411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:03.780439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:03.781340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:03.781340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:03.781340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:03.785226 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 295/592] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:03.786076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc56f0-1edb-40dc-9986-cf2c05f93c9b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:03.808986 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:03.812553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd94b05-5eb7-4a07-9b37-3b723786ab11 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:03.850741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b4343-88d4-4046-9590-f38735f46460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:03.851461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b4343-88d4-4046-9590-f38735f46460 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:03.851751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b4343-88d4-4046-9590-f38735f46460 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:03.852016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b4343-88d4-4046-9590-f38735f46460 None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:03.852739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b4343-88d4-4046-9590-f38735f46460 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:03.853036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b4343-88d4-4046-9590-f38735f46460 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:03.854459 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b4343-88d4-4046-9590-f38735f46460 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:03.872240 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 298/593] 10.222.0.27 () {56 vars in 1139 bytes} [Thu Sep 8 15:55:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:03.877609 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b4343-88d4-4046-9590-f38735f46460 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:03.893654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc56f0-1edb-40dc-9986-cf2c05f93c9b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:03.909163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:03.931168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7d292-2903-4244-9ea9-7f750f5457e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:03.932129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7d292-2903-4244-9ea9-7f750f5457e6 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:03.932557 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7d292-2903-4244-9ea9-7f750f5457e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:03.932963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7d292-2903-4244-9ea9-7f750f5457e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:03.933804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7d292-2903-4244-9ea9-7f750f5457e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:03.934299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7d292-2903-4244-9ea9-7f750f5457e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:03.935758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7d292-2903-4244-9ea9-7f750f5457e6 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:03.950007 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 296/594] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:03.950878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7d292-2903-4244-9ea9-7f750f5457e6 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:03.988931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b4343-88d4-4046-9590-f38735f46460 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:04.032037 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbbc47-090f-4adc-ad2b-fe6399b45a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:04.032674 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbbc47-090f-4adc-ad2b-fe6399b45a0e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:04.032942 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbbc47-090f-4adc-ad2b-fe6399b45a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:04.033203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbbc47-090f-4adc-ad2b-fe6399b45a0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:04.040826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbbc47-090f-4adc-ad2b-fe6399b45a0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:04.042260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbbc47-090f-4adc-ad2b-fe6399b45a0e None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:04.054267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-57dbbc47-090f-4adc-ad2b-fe6399b45a0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:04.054267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:04.054267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:04.054267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:04.054267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:04.054267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:04.054267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:04.054267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:04.054267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:04.054267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:04.054267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:04.054267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:04.054267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:04.054267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:04.054267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:04.055207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:04.056578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:04.056578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:04.056578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:04.056578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:04.056578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:04.056578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:04.056578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:04.056578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:04.056578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 299/595] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:04.056578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbbc47-090f-4adc-ad2b-fe6399b45a0e None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:04.065442 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7d292-2903-4244-9ea9-7f750f5457e6 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:04.075372 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:04.102152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c869a10-ffcb-4c35-9b54-aa3ec2146664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:04.102768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c869a10-ffcb-4c35-9b54-aa3ec2146664 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:04.103103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c869a10-ffcb-4c35-9b54-aa3ec2146664 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:04.103365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c869a10-ffcb-4c35-9b54-aa3ec2146664 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:04.103984 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c869a10-ffcb-4c35-9b54-aa3ec2146664 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:04.104285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c869a10-ffcb-4c35-9b54-aa3ec2146664 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:04.106233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c869a10-ffcb-4c35-9b54-aa3ec2146664 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:04.118736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 297/596] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:04.118736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c869a10-ffcb-4c35-9b54-aa3ec2146664 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:04.160519 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:04.165037 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbbc47-090f-4adc-ad2b-fe6399b45a0e None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:04.201385 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a46df8-6537-4d48-b861-6627cf13ed8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:04.202039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a46df8-6537-4d48-b861-6627cf13ed8c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:04.202310 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a46df8-6537-4d48-b861-6627cf13ed8c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:04.202578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a46df8-6537-4d48-b861-6627cf13ed8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:04.216121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a46df8-6537-4d48-b861-6627cf13ed8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:04.217598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a46df8-6537-4d48-b861-6627cf13ed8c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-a8a46df8-6537-4d48-b861-6627cf13ed8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:04.232160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:04.233239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:04.234140 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:04.234140 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:04.234140 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:04.234140 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 300/597] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:04.234140 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a46df8-6537-4d48-b861-6627cf13ed8c None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:04.243856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c869a10-ffcb-4c35-9b54-aa3ec2146664 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:04.282364 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f46b09-48bd-473b-8e16-316cd9665c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:04.283130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f46b09-48bd-473b-8e16-316cd9665c06 None admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:04.283413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f46b09-48bd-473b-8e16-316cd9665c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:04.283793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f46b09-48bd-473b-8e16-316cd9665c06 None admin] PATH_INFO: `/v3/projects/a825a0b1238d4382995da440944f453b/users/89df988d73ff48f3b7a92d335df9b633/roles/b256252b45e541a491376a134f502183` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:04.299884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f46b09-48bd-473b-8e16-316cd9665c06 None admin] RBAC: Authorizing `identity:create_grant(project_id=a825a0b1238d4382995da440944f453b, user_id=89df988d73ff48f3b7a92d335df9b633, role_id=b256252b45e541a491376a134f502183)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:04.301473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f46b09-48bd-473b-8e16-316cd9665c06 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:04.302025 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:04.316299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 298/598] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:55:04 2022] PUT /identity/v3/projects/a825a0b1238d4382995da440944f453b/users/89df988d73ff48f3b7a92d335df9b633/roles/b256252b45e541a491376a134f502183 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:55:04.317122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f46b09-48bd-473b-8e16-316cd9665c06 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:04.351256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a46df8-6537-4d48-b861-6627cf13ed8c None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:04.400769 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01539b35-1f77-4bd0-81f1-b34faa612cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:04.401784 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01539b35-1f77-4bd0-81f1-b34faa612cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:04.402110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01539b35-1f77-4bd0-81f1-b34faa612cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:04.402431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01539b35-1f77-4bd0-81f1-b34faa612cf2 None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:04.403423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01539b35-1f77-4bd0-81f1-b34faa612cf2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:04.403776 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01539b35-1f77-4bd0-81f1-b34faa612cf2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:04.405933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01539b35-1f77-4bd0-81f1-b34faa612cf2 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:04.423497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 301/599] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:04.424417 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01539b35-1f77-4bd0-81f1-b34faa612cf2 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:04.438055 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:04.455654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f46b09-48bd-473b-8e16-316cd9665c06 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:04.494858 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6a8b9-0278-4d1d-8dc8-45ec50debea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:04.495546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6a8b9-0278-4d1d-8dc8-45ec50debea8 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:04.495827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6a8b9-0278-4d1d-8dc8-45ec50debea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:04.496101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6a8b9-0278-4d1d-8dc8-45ec50debea8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:04.504190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6a8b9-0278-4d1d-8dc8-45ec50debea8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:04.505654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6a8b9-0278-4d1d-8dc8-45ec50debea8 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:04.518299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-2bf6a8b9-0278-4d1d-8dc8-45ec50debea8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:04.518299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:04.518299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:04.518299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:04.518299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:04.518299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:04.518299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:04.518299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:04.518299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:04.518299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:04.518299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:04.518299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:04.518299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:04.518299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:04.518299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:04.519254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:04.520625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:04.520625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:04.520625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:04.520625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:04.520625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:04.520625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:04.520625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:04.520625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:04.520625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 299/600] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:04.520625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6a8b9-0278-4d1d-8dc8-45ec50debea8 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:04.548290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01539b35-1f77-4bd0-81f1-b34faa612cf2 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:04.579848 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:04.585902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1b98e-da7d-496e-ba48-2007ce1fbf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:04.586794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1b98e-da7d-496e-ba48-2007ce1fbf1c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:04.587608 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1b98e-da7d-496e-ba48-2007ce1fbf1c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:04.588015 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1b98e-da7d-496e-ba48-2007ce1fbf1c None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:04.588800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1b98e-da7d-496e-ba48-2007ce1fbf1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:04.589284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1b98e-da7d-496e-ba48-2007ce1fbf1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:04.591028 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1b98e-da7d-496e-ba48-2007ce1fbf1c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:04.607604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 302/601] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:04.608840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1b98e-da7d-496e-ba48-2007ce1fbf1c None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:04.639467 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6a8b9-0278-4d1d-8dc8-45ec50debea8 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:04.677602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc4c91-fb29-47a8-a571-902bb3fdd12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:04.678475 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc4c91-fb29-47a8-a571-902bb3fdd12c None admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:04.678849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc4c91-fb29-47a8-a571-902bb3fdd12c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:04.679213 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc4c91-fb29-47a8-a571-902bb3fdd12c None admin] PATH_INFO: `/v3/projects/a825a0b1238d4382995da440944f453b/users/6e9209e77ce44b128e25a8ddcf78f8b5/roles/b256252b45e541a491376a134f502183` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:04.698095 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc4c91-fb29-47a8-a571-902bb3fdd12c None admin] RBAC: Authorizing `identity:create_grant(project_id=a825a0b1238d4382995da440944f453b, user_id=6e9209e77ce44b128e25a8ddcf78f8b5, role_id=b256252b45e541a491376a134f502183)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:04.700975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bc4c91-fb29-47a8-a571-902bb3fdd12c None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:04.705019 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:04.721401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1b98e-da7d-496e-ba48-2007ce1fbf1c None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:04.735492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 300/602] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:55:04 2022] PUT /identity/v3/projects/a825a0b1238d4382995da440944f453b/users/6e9209e77ce44b128e25a8ddcf78f8b5/roles/b256252b45e541a491376a134f502183 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:55:04.735492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc4c91-fb29-47a8-a571-902bb3fdd12c None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:04.800143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3ddc5-fbe4-476d-b7d8-84a33f3024bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:04.800689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3ddc5-fbe4-476d-b7d8-84a33f3024bd None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:04.800959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3ddc5-fbe4-476d-b7d8-84a33f3024bd None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:04.801224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3ddc5-fbe4-476d-b7d8-84a33f3024bd None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:04.801931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3ddc5-fbe4-476d-b7d8-84a33f3024bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:04.802229 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3ddc5-fbe4-476d-b7d8-84a33f3024bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:04.806244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3ddc5-fbe4-476d-b7d8-84a33f3024bd None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:04.840913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 303/603] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:04.843150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3ddc5-fbe4-476d-b7d8-84a33f3024bd None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:04.906316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:04.958714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc4c91-fb29-47a8-a571-902bb3fdd12c None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:04.967201 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3ddc5-fbe4-476d-b7d8-84a33f3024bd None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:04.997619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176cf75-0885-4581-af47-7c9ac2003b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:04.998784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176cf75-0885-4581-af47-7c9ac2003b6b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:04.999262 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176cf75-0885-4581-af47-7c9ac2003b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:04.999756 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176cf75-0885-4581-af47-7c9ac2003b6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:05.003543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48799df6-f596-4763-9b57-6fdad851eda9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:05.004040 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48799df6-f596-4763-9b57-6fdad851eda9 None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:05.004316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48799df6-f596-4763-9b57-6fdad851eda9 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:05.004584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48799df6-f596-4763-9b57-6fdad851eda9 None admin] PATH_INFO: `/v3/projects/a825a0b1238d4382995da440944f453b/users/f24ee16c043244938901c4e850dea967/roles/b256252b45e541a491376a134f502183` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:05.009360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176cf75-0885-4581-af47-7c9ac2003b6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:05.011059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176cf75-0885-4581-af47-7c9ac2003b6b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:05.021301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48799df6-f596-4763-9b57-6fdad851eda9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a825a0b1238d4382995da440944f453b, user_id=f24ee16c043244938901c4e850dea967, role_id=b256252b45e541a491376a134f502183)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:05.022750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48799df6-f596-4763-9b57-6fdad851eda9 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:05.024242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-7176cf75-0885-4581-af47-7c9ac2003b6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:05.024242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:05.024242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:05.024242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:05.024242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:05.024242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:05.024242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:05.024242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:05.024242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:05.024242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:05.024242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:05.024242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:05.024242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:05.024242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:05.024242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:05.025147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:05.026075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:05.026075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:05.026075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:05.026075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:05.026075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:05.026075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:05.026075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:05.026075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:05.028704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 301/604] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:05.034573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176cf75-0885-4581-af47-7c9ac2003b6b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:05.044661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 304/605] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:55:04 2022] PUT /identity/v3/projects/a825a0b1238d4382995da440944f453b/users/f24ee16c043244938901c4e850dea967/roles/b256252b45e541a491376a134f502183 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 15:55:05.102904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:05.160073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176cf75-0885-4581-af47-7c9ac2003b6b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:05.198648 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de3fff9-0fe3-4c13-bfc4-ebda3beb093e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:05.199760 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3fff9-0fe3-4c13-bfc4-ebda3beb093e None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:05.200357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3fff9-0fe3-4c13-bfc4-ebda3beb093e None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:05.200826 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3fff9-0fe3-4c13-bfc4-ebda3beb093e None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:05.201728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3fff9-0fe3-4c13-bfc4-ebda3beb093e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:05.202254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3fff9-0fe3-4c13-bfc4-ebda3beb093e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:05.204032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de3fff9-0fe3-4c13-bfc4-ebda3beb093e None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:05.219067 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 302/606] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:05.226547 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48799df6-f596-4763-9b57-6fdad851eda9 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:05.248225 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606a9c0-67fe-421b-9960-7789bc048270 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:05.248591 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606a9c0-67fe-421b-9960-7789bc048270 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:05.248762 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606a9c0-67fe-421b-9960-7789bc048270 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:05.250118 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 303/607] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:05.256736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e523851-e183-442d-83ce-1f240fd31a2f None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:05.257174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e523851-e183-442d-83ce-1f240fd31a2f None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:05.257560 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e523851-e183-442d-83ce-1f240fd31a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:05.279030 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-1e523851-e183-442d-83ce-1f240fd31a2f None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:05.280521 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1e523851-e183-442d-83ce-1f240fd31a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:05.360714 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:05.366294 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48799df6-f596-4763-9b57-6fdad851eda9 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:05.372644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 304/608] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:55:05.422903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb05a5c-5d12-4b05-a3eb-d2fe859a0c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:05.423629 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb05a5c-5d12-4b05-a3eb-d2fe859a0c0a None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:05.423907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb05a5c-5d12-4b05-a3eb-d2fe859a0c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:05.424177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb05a5c-5d12-4b05-a3eb-d2fe859a0c0a None admin] PATH_INFO: `/v3/projects/a825a0b1238d4382995da440944f453b/users/3aae65f9ec394f97aeafad43932dc49c/roles/b256252b45e541a491376a134f502183` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:05.443780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb05a5c-5d12-4b05-a3eb-d2fe859a0c0a None admin] RBAC: Authorizing `identity:create_grant(project_id=a825a0b1238d4382995da440944f453b, user_id=3aae65f9ec394f97aeafad43932dc49c, role_id=b256252b45e541a491376a134f502183)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:05.445585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb05a5c-5d12-4b05-a3eb-d2fe859a0c0a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:05.464550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 305/609] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:55:05 2022] PUT /identity/v3/projects/a825a0b1238d4382995da440944f453b/users/3aae65f9ec394f97aeafad43932dc49c/roles/b256252b45e541a491376a134f502183 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:55:05.667840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e523851-e183-442d-83ce-1f240fd31a2f None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:05.730312 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:05.796636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e523851-e183-442d-83ce-1f240fd31a2f None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:05.835563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411490ae-30fe-40c3-b9af-be49cf635fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:05.836209 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411490ae-30fe-40c3-b9af-be49cf635fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:05.836484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411490ae-30fe-40c3-b9af-be49cf635fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:05.836751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411490ae-30fe-40c3-b9af-be49cf635fe6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:05.847537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411490ae-30fe-40c3-b9af-be49cf635fe6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-411490ae-30fe-40c3-b9af-be49cf635fe6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:05.852316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:05.853288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:05.854145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:05.854145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:05.854145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:05.854145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:05.854145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:05.854145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:05.854145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:05.854145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:05.854145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:05.854145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:05.854145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:05.854145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:05.854145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 305/610] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:55:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:05.861375 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb05a5c-5d12-4b05-a3eb-d2fe859a0c0a None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:06.005686 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb05a5c-5d12-4b05-a3eb-d2fe859a0c0a None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:06.057280 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c8b03-a1bd-4081-928b-7de404bb67ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:06.058289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c8b03-a1bd-4081-928b-7de404bb67ac None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:06.058787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c8b03-a1bd-4081-928b-7de404bb67ac None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:06.059268 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c8b03-a1bd-4081-928b-7de404bb67ac None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:06.060162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c8b03-a1bd-4081-928b-7de404bb67ac None admin] RBAC: Adding query filter params (name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:06.060675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c8b03-a1bd-4081-928b-7de404bb67ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:06.062331 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c8b03-a1bd-4081-928b-7de404bb67ac None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:06.074826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 306/611] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:55:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:06.080568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411490ae-30fe-40c3-b9af-be49cf635fe6 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:06.213785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411490ae-30fe-40c3-b9af-be49cf635fe6 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:06.216157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:06.248491 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6da184-3d32-4600-9db8-4e0cf9cd4a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:06.249127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6da184-3d32-4600-9db8-4e0cf9cd4a16 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:06.249401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6da184-3d32-4600-9db8-4e0cf9cd4a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:06.249671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6da184-3d32-4600-9db8-4e0cf9cd4a16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:06.259946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6da184-3d32-4600-9db8-4e0cf9cd4a16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:06.261349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6da184-3d32-4600-9db8-4e0cf9cd4a16 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-db6da184-3d32-4600-9db8-4e0cf9cd4a16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:06.276638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:06.277657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:06.278590 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:06.278590 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:06.278590 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:06.278590 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 306/612] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:06.283754 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c8b03-a1bd-4081-928b-7de404bb67ac None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:06.296770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:06.408842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c8b03-a1bd-4081-928b-7de404bb67ac None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:06.452710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5764df8-c710-41e1-8f39-2e10c2b5debd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:06.453326 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5764df8-c710-41e1-8f39-2e10c2b5debd None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:06.453599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5764df8-c710-41e1-8f39-2e10c2b5debd None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:06.453867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5764df8-c710-41e1-8f39-2e10c2b5debd None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:06.454510 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5764df8-c710-41e1-8f39-2e10c2b5debd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:06.454815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5764df8-c710-41e1-8f39-2e10c2b5debd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:06.456365 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5764df8-c710-41e1-8f39-2e10c2b5debd None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:06.475243 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 307/613] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:06.477394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6da184-3d32-4600-9db8-4e0cf9cd4a16 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:06.601510 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6da184-3d32-4600-9db8-4e0cf9cd4a16 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:06.637854 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa148af4-8c3d-484a-a017-6b4e560ec50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:06.638501 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa148af4-8c3d-484a-a017-6b4e560ec50f None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:06.638778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa148af4-8c3d-484a-a017-6b4e560ec50f None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:06.639051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa148af4-8c3d-484a-a017-6b4e560ec50f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:06.646939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa148af4-8c3d-484a-a017-6b4e560ec50f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:06.648421 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa148af4-8c3d-484a-a017-6b4e560ec50f None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-fa148af4-8c3d-484a-a017-6b4e560ec50f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:06.659695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:06.660678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:06.661638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:06.661638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:06.661638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:06.661638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:06.661638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:06.661638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:06.661638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:06.661638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:06.661638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 15:55:06.661638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:06.661638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 307/614] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:55:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:06.666890 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5764df8-c710-41e1-8f39-2e10c2b5debd None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:06.752958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:06.789426 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5764df8-c710-41e1-8f39-2e10c2b5debd None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:06.832869 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4489fa2-d0ac-48cb-bec4-b3a356c63fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:06.833491 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4489fa2-d0ac-48cb-bec4-b3a356c63fef None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:06.833773 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4489fa2-d0ac-48cb-bec4-b3a356c63fef None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:06.834033 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4489fa2-d0ac-48cb-bec4-b3a356c63fef None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:06.834778 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4489fa2-d0ac-48cb-bec4-b3a356c63fef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:06.835085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4489fa2-d0ac-48cb-bec4-b3a356c63fef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:06.836556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4489fa2-d0ac-48cb-bec4-b3a356c63fef None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:06.864873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 308/615] 10.222.0.27 () {56 vars in 1135 bytes} [Thu Sep 8 15:55:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:06.870781 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa148af4-8c3d-484a-a017-6b4e560ec50f None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:06.915781 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:07.042624 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa148af4-8c3d-484a-a017-6b4e560ec50f None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:07.093235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369d527-e97a-42ef-bf32-dad6bbf8cddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:07.093235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369d527-e97a-42ef-bf32-dad6bbf8cddd None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:07.093235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369d527-e97a-42ef-bf32-dad6bbf8cddd None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:07.093235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6369d527-e97a-42ef-bf32-dad6bbf8cddd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:07.098726 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d11ee-52bd-4d01-883e-f9760cd4a83a None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:07.099227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d11ee-52bd-4d01-883e-f9760cd4a83a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:07.099480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d11ee-52bd-4d01-883e-f9760cd4a83a None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:07.101814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 309/616] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:07.105729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369d527-e97a-42ef-bf32-dad6bbf8cddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:07.106743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764c173-d559-4b01-a78e-300633ec6d82 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:07.107026 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764c173-d559-4b01-a78e-300633ec6d82 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:07.107295 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5764c173-d559-4b01-a78e-300633ec6d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:07.107572 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6369d527-e97a-42ef-bf32-dad6bbf8cddd None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-6369d527-e97a-42ef-bf32-dad6bbf8cddd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:07.129323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:07.130311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:07.131203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:07.131203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:07.131203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:07.131203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 308/617] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:07.137738 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369d527-e97a-42ef-bf32-dad6bbf8cddd None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:07.141231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-5764c173-d559-4b01-a78e-300633ec6d82 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:07.142890 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5764c173-d559-4b01-a78e-300633ec6d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:07.208683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:07.244516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6369d527-e97a-42ef-bf32-dad6bbf8cddd None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:07.287913 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630636a-f683-4582-9093-d27720414b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:07.288534 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630636a-f683-4582-9093-d27720414b3d None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:07.288807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630636a-f683-4582-9093-d27720414b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:07.289073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630636a-f683-4582-9093-d27720414b3d None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:07.291699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630636a-f683-4582-9093-d27720414b3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:07.291699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630636a-f683-4582-9093-d27720414b3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:07.291699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630636a-f683-4582-9093-d27720414b3d None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:07.305568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 309/618] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:07.310001 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 310/619] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:55:07.311824 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630636a-f683-4582-9093-d27720414b3d None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:07.415288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630636a-f683-4582-9093-d27720414b3d None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:07.448339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:07.453490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ad6cb-4b5c-48ae-9c42-4dbbceb98781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:07.454149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ad6cb-4b5c-48ae-9c42-4dbbceb98781 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:07.454511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ad6cb-4b5c-48ae-9c42-4dbbceb98781 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:07.454681 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ad6cb-4b5c-48ae-9c42-4dbbceb98781 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:07.465121 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ad6cb-4b5c-48ae-9c42-4dbbceb98781 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:07.466837 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493ad6cb-4b5c-48ae-9c42-4dbbceb98781 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:07.479635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-493ad6cb-4b5c-48ae-9c42-4dbbceb98781 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:07.479635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:07.479635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:07.479635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:07.479635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:07.479635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:07.479635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:07.479635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:07.479635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:07.479635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:07.479635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:07.479635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:07.479635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:07.479635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:07.479635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:07.481761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:07.483807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:07.483807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:07.483807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:07.483807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:07.483807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:07.483807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:07.483807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:07.483807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:07.483807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 310/620] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:07.496701 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764c173-d559-4b01-a78e-300633ec6d82 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:07.586256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:07.619244 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8f6ad-f235-4602-9e61-c47a9ba8e600 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:07.620264 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8f6ad-f235-4602-9e61-c47a9ba8e600 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:07.620920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8f6ad-f235-4602-9e61-c47a9ba8e600 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:07.622726 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 311/621] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:55:07.629841 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8cb8f-cf4f-46ec-9505-0870f92570fe None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:07.630455 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8cb8f-cf4f-46ec-9505-0870f92570fe None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:07.630940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8cb8f-cf4f-46ec-9505-0870f92570fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:07.651801 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5764c173-d559-4b01-a78e-300633ec6d82 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:07.652506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-5ed8cb8f-cf4f-46ec-9505-0870f92570fe None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:07.653972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5ed8cb8f-cf4f-46ec-9505-0870f92570fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:07.733131 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827d61e-2b4c-4c65-bcd6-1a8ee6d3cad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:07.733669 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827d61e-2b4c-4c65-bcd6-1a8ee6d3cad4 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:07.733954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827d61e-2b4c-4c65-bcd6-1a8ee6d3cad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:07.734219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0827d61e-2b4c-4c65-bcd6-1a8ee6d3cad4 None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:07.734924 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827d61e-2b4c-4c65-bcd6-1a8ee6d3cad4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:07.735409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827d61e-2b4c-4c65-bcd6-1a8ee6d3cad4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:07.736704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 312/622] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:55:07.736865 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0827d61e-2b4c-4c65-bcd6-1a8ee6d3cad4 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:07.753654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8cb8f-cf4f-46ec-9505-0870f92570fe None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:07.758798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 311/623] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:07.769700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827d61e-2b4c-4c65-bcd6-1a8ee6d3cad4 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:07.928258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8cb8f-cf4f-46ec-9505-0870f92570fe None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:07.947670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0827d61e-2b4c-4c65-bcd6-1a8ee6d3cad4 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:07.948032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:07.971466 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:07.974075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2abaf-ceb7-4c81-81a6-fd0b7c2fb95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:07.974803 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2abaf-ceb7-4c81-81a6-fd0b7c2fb95a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:07.975076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2abaf-ceb7-4c81-81a6-fd0b7c2fb95a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:07.975341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2abaf-ceb7-4c81-81a6-fd0b7c2fb95a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:07.988597 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2abaf-ceb7-4c81-81a6-fd0b7c2fb95a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-1ad2abaf-ceb7-4c81-81a6-fd0b7c2fb95a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:07.993391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:07.994317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:07.995190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:07.995190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:07.995190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:07.995190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:07.995190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:07.995190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:07.995190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:07.995190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:07.995190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:07.995190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:07.995190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:07.995190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:07.998702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ebebc0-1447-4dbe-8c8c-e0f6c2759ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:07.999380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 313/624] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:55:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:07.999831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebebc0-1447-4dbe-8c8c-e0f6c2759ea2 None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:08.000267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebebc0-1447-4dbe-8c8c-e0f6c2759ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:08.000664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c2cdb-a54d-4de5-bdc5-149739d698d5 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:08.000831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ebebc0-1447-4dbe-8c8c-e0f6c2759ea2 None admin] PATH_INFO: `/v3/projects/a825a0b1238d4382995da440944f453b/users/de54687547a14ab89cc41a1fabaab400/roles/b256252b45e541a491376a134f502183` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:08.000934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c2cdb-a54d-4de5-bdc5-149739d698d5 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:08.001202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c2cdb-a54d-4de5-bdc5-149739d698d5 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:08.003383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 314/625] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:08.004117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c2cdb-a54d-4de5-bdc5-149739d698d5 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:08.019385 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebebc0-1447-4dbe-8c8c-e0f6c2759ea2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a825a0b1238d4382995da440944f453b, user_id=de54687547a14ab89cc41a1fabaab400, role_id=b256252b45e541a491376a134f502183)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:08.021696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ebebc0-1447-4dbe-8c8c-e0f6c2759ea2 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:08.041828 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 312/626] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:55:07 2022] PUT /identity/v3/projects/a825a0b1238d4382995da440944f453b/users/de54687547a14ab89cc41a1fabaab400/roles/b256252b45e541a491376a134f502183 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:55:08.043807 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b8709-996d-4d38-b60c-e0c69fed7238 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:08.044299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b8709-996d-4d38-b60c-e0c69fed7238 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:08.044794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b8709-996d-4d38-b60c-e0c69fed7238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:08.069174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-528b8709-996d-4d38-b60c-e0c69fed7238 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:08.070848 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-528b8709-996d-4d38-b60c-e0c69fed7238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:08.140204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c2cdb-a54d-4de5-bdc5-149739d698d5 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:08.160329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 313/627] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:55:08.161433 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b8709-996d-4d38-b60c-e0c69fed7238 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:08.183416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43582e5b-1945-4929-9f39-6d2fc5a6a231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:08.196168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43582e5b-1945-4929-9f39-6d2fc5a6a231 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:08.196168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43582e5b-1945-4929-9f39-6d2fc5a6a231 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:08.196168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43582e5b-1945-4929-9f39-6d2fc5a6a231 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:08.196168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43582e5b-1945-4929-9f39-6d2fc5a6a231 None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:08.196168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43582e5b-1945-4929-9f39-6d2fc5a6a231 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:08.196168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43582e5b-1945-4929-9f39-6d2fc5a6a231 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:08.200226 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 315/628] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:55:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:08.208516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43582e5b-1945-4929-9f39-6d2fc5a6a231 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:08.285775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:08.319578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:08.324085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b8709-996d-4d38-b60c-e0c69fed7238 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:08.361896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a831901-6c8d-4c25-aa29-2b1590d1f244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:08.362999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a831901-6c8d-4c25-aa29-2b1590d1f244 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:08.363634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a831901-6c8d-4c25-aa29-2b1590d1f244 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:08.364674 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a831901-6c8d-4c25-aa29-2b1590d1f244 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:08.368785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43582e5b-1945-4929-9f39-6d2fc5a6a231 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:08.378451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a831901-6c8d-4c25-aa29-2b1590d1f244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-4a831901-6c8d-4c25-aa29-2b1590d1f244 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:08.385401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:08.386566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:08.387635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:08.387635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:08.387635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:08.387635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:08.387635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:08.387635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:08.387635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:08.387635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:08.387635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:08.387635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:08.387635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:08.387635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:08.395943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 314/629] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:55:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:08.395943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a831901-6c8d-4c25-aa29-2b1590d1f244 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:08.434473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d18685-911c-42b4-8938-8c0eb43c8fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:08.435375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d18685-911c-42b4-8938-8c0eb43c8fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:08.435701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d18685-911c-42b4-8938-8c0eb43c8fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:08.436209 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d18685-911c-42b4-8938-8c0eb43c8fa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:08.452337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d18685-911c-42b4-8938-8c0eb43c8fa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:08.454588 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d18685-911c-42b4-8938-8c0eb43c8fa2 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-70d18685-911c-42b4-8938-8c0eb43c8fa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:08.473038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:08.474031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:08.474928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:08.474928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:08.474928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:08.474928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 316/630] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:08.475908 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6e049-5da0-462e-ac3b-afa7c339ca58 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:08.476098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6e049-5da0-462e-ac3b-afa7c339ca58 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:08.479480 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6e049-5da0-462e-ac3b-afa7c339ca58 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:08.479480 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 317/631] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:08.483548 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6e049-5da0-462e-ac3b-afa7c339ca58 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:08.521312 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a831901-6c8d-4c25-aa29-2b1590d1f244 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:08.557185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ce371-5ef8-4ac5-a701-8582904c10b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:08.557748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ce371-5ef8-4ac5-a701-8582904c10b5 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:08.558017 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ce371-5ef8-4ac5-a701-8582904c10b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:08.558261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125ce371-5ef8-4ac5-a701-8582904c10b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:08.558948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ce371-5ef8-4ac5-a701-8582904c10b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:08.559246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ce371-5ef8-4ac5-a701-8582904c10b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:08.560657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125ce371-5ef8-4ac5-a701-8582904c10b5 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:08.569440 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 315/632] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:55:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:08.571184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ce371-5ef8-4ac5-a701-8582904c10b5 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:08.589261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:08.600054 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6e049-5da0-462e-ac3b-afa7c339ca58 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:08.600639 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:08.636368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37edd99e-8fb3-4c05-93db-e27db9f16503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:08.636982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edd99e-8fb3-4c05-93db-e27db9f16503 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:08.637257 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edd99e-8fb3-4c05-93db-e27db9f16503 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:08.637532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37edd99e-8fb3-4c05-93db-e27db9f16503 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:08.638178 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37edd99e-8fb3-4c05-93db-e27db9f16503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:08.638477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37edd99e-8fb3-4c05-93db-e27db9f16503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:08.639889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37edd99e-8fb3-4c05-93db-e27db9f16503 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:08.652503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 318/633] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:08.653723 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d9404-afbc-4bd1-8703-4cd538a73162 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:08.654003 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d9404-afbc-4bd1-8703-4cd538a73162 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:08.654273 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d9404-afbc-4bd1-8703-4cd538a73162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:08.673842 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-042d9404-afbc-4bd1-8703-4cd538a73162 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:08.675179 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-042d9404-afbc-4bd1-8703-4cd538a73162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:08.686726 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125ce371-5ef8-4ac5-a701-8582904c10b5 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:08.721859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef249800-c1ff-4585-aa42-ee052b889716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:08.722501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef249800-c1ff-4585-aa42-ee052b889716 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:08.722771 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef249800-c1ff-4585-aa42-ee052b889716 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:08.723037 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef249800-c1ff-4585-aa42-ee052b889716 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:08.733910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef249800-c1ff-4585-aa42-ee052b889716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-ef249800-c1ff-4585-aa42-ee052b889716 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:08.739939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:08.740904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:08.741779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:08.741779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:08.741779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:08.741779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:08.741779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:08.741779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:08.741779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:08.741779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:08.741779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:08.741779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:08.741779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:08.741779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:08.741779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 316/634] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:55:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:08.742482 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef249800-c1ff-4585-aa42-ee052b889716 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:08.751619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 319/635] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:55:08.752562 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d9404-afbc-4bd1-8703-4cd538a73162 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:08.842612 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:08.862594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:08.863072 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d9404-afbc-4bd1-8703-4cd538a73162 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:08.891231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef249800-c1ff-4585-aa42-ee052b889716 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:08.900200 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65837a84-9dc6-496a-b3d8-13bba6f78a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:08.900846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65837a84-9dc6-496a-b3d8-13bba6f78a48 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:08.901123 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65837a84-9dc6-496a-b3d8-13bba6f78a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:08.901393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65837a84-9dc6-496a-b3d8-13bba6f78a48 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:08.909229 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65837a84-9dc6-496a-b3d8-13bba6f78a48 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:08.910693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65837a84-9dc6-496a-b3d8-13bba6f78a48 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-65837a84-9dc6-496a-b3d8-13bba6f78a48 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:08.927187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:08.928252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:08.929220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:08.929220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:08.929220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:08.929220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:08.929220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:08.929220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:08.929220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:08.929220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:08.929220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 15:55:08.929220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:08.931848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 320/636] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:55:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:08.932790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65837a84-9dc6-496a-b3d8-13bba6f78a48 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:08.943555 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163ed59-21e2-4f16-b57d-a83b86ab9cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:08.944667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163ed59-21e2-4f16-b57d-a83b86ab9cbe None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:08.945135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163ed59-21e2-4f16-b57d-a83b86ab9cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:08.945666 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d163ed59-21e2-4f16-b57d-a83b86ab9cbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:08.959563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163ed59-21e2-4f16-b57d-a83b86ab9cbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:08.962612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d163ed59-21e2-4f16-b57d-a83b86ab9cbe None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-d163ed59-21e2-4f16-b57d-a83b86ab9cbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:08.986600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:08.987662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:08.988579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:08.988579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:08.988579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:08.991490 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 317/637] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:08.992546 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163ed59-21e2-4f16-b57d-a83b86ab9cbe None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:09.042855 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65837a84-9dc6-496a-b3d8-13bba6f78a48 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:09.075066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cab03-5733-4447-9030-020cc90d23b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:09.075746 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cab03-5733-4447-9030-020cc90d23b3 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:09.075990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cab03-5733-4447-9030-020cc90d23b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:09.076280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cab03-5733-4447-9030-020cc90d23b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:09.076910 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cab03-5733-4447-9030-020cc90d23b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:09.077210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cab03-5733-4447-9030-020cc90d23b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:09.078626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cab03-5733-4447-9030-020cc90d23b3 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:09.088578 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 321/638] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:55:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:09.089464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cab03-5733-4447-9030-020cc90d23b3 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:09.094594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:09.118340 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d163ed59-21e2-4f16-b57d-a83b86ab9cbe None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:09.155814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:09.161768 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876bac2-5815-4076-a1bf-e4ee20129f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:09.163041 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876bac2-5815-4076-a1bf-e4ee20129f92 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:09.164221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876bac2-5815-4076-a1bf-e4ee20129f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:09.164221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876bac2-5815-4076-a1bf-e4ee20129f92 None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:09.164767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876bac2-5815-4076-a1bf-e4ee20129f92 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:09.165483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876bac2-5815-4076-a1bf-e4ee20129f92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:09.167018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876bac2-5815-4076-a1bf-e4ee20129f92 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:09.194156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 318/639] 10.222.0.27 () {56 vars in 1137 bytes} [Thu Sep 8 15:55:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:09.195562 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876bac2-5815-4076-a1bf-e4ee20129f92 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:09.197733 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cab03-5733-4447-9030-020cc90d23b3 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:09.235042 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae5ff6-0136-40a5-8bda-fdf4d84941dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:09.235764 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae5ff6-0136-40a5-8bda-fdf4d84941dd None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:09.236180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae5ff6-0136-40a5-8bda-fdf4d84941dd None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:09.236597 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae5ff6-0136-40a5-8bda-fdf4d84941dd None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:09.237335 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae5ff6-0136-40a5-8bda-fdf4d84941dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:09.237636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae5ff6-0136-40a5-8bda-fdf4d84941dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:09.239091 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae5ff6-0136-40a5-8bda-fdf4d84941dd None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:09.257585 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 322/640] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:09.258232 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae5ff6-0136-40a5-8bda-fdf4d84941dd None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:09.302568 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876bac2-5815-4076-a1bf-e4ee20129f92 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:09.324323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:09.338931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1d93a-d89f-4205-9cd2-0cf4410b3b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:09.339588 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1d93a-d89f-4205-9cd2-0cf4410b3b00 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:09.339863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1d93a-d89f-4205-9cd2-0cf4410b3b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:09.340127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c1d93a-d89f-4205-9cd2-0cf4410b3b00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:09.350314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c1d93a-d89f-4205-9cd2-0cf4410b3b00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-10c1d93a-d89f-4205-9cd2-0cf4410b3b00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:09.355064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:09.356071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:09.356926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:09.356926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:09.356926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:09.356926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:09.356926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:09.356926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:09.356926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:09.356926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:09.356926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:09.356926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:09.356926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:09.356926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:09.356926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 319/641] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:55:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:09.357623 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1d93a-d89f-4205-9cd2-0cf4410b3b00 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:09.369160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae5ff6-0136-40a5-8bda-fdf4d84941dd None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:09.404661 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638e582-296d-4d05-b9a7-fa0626746948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:09.405297 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638e582-296d-4d05-b9a7-fa0626746948 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:09.405730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638e582-296d-4d05-b9a7-fa0626746948 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:09.405863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a638e582-296d-4d05-b9a7-fa0626746948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:09.409130 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:09.420746 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638e582-296d-4d05-b9a7-fa0626746948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:09.422193 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a638e582-296d-4d05-b9a7-fa0626746948 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-a638e582-296d-4d05-b9a7-fa0626746948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:09.437375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:09.438623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:09.439863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:09.439863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:09.439863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:09.443206 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 323/642] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:09.444083 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638e582-296d-4d05-b9a7-fa0626746948 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:09.464910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c1d93a-d89f-4205-9cd2-0cf4410b3b00 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:09.500315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558d5e9-d527-47f8-9779-baaaad199d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:09.501026 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558d5e9-d527-47f8-9779-baaaad199d61 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:09.501268 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558d5e9-d527-47f8-9779-baaaad199d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:09.501453 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558d5e9-d527-47f8-9779-baaaad199d61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:09.550965 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a638e582-296d-4d05-b9a7-fa0626746948 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:09.571054 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:09.575662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558d5e9-d527-47f8-9779-baaaad199d61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:09.577130 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5558d5e9-d527-47f8-9779-baaaad199d61 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:09.593743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e6fdb-15cb-40df-9ca1-8de846c135c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:09.594531 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e6fdb-15cb-40df-9ca1-8de846c135c0 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:09.595093 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e6fdb-15cb-40df-9ca1-8de846c135c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:09.595368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0e6fdb-15cb-40df-9ca1-8de846c135c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:09.605140 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e6fdb-15cb-40df-9ca1-8de846c135c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:09.607293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0e6fdb-15cb-40df-9ca1-8de846c135c0 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-1d0e6fdb-15cb-40df-9ca1-8de846c135c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:09.621040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:09.621997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:09.623094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:09.623094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:09.623094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:09.623094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:09.623094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:09.623094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:09.623094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:09.623094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:09.623094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:55:09.623094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:09.623728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 324/643] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Sep 8 15:55:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:09.624386 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e6fdb-15cb-40df-9ca1-8de846c135c0 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:09.628554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:09.686810 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 320/644] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:55:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:55:09.686810 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558d5e9-d527-47f8-9779-baaaad199d61 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:09.735539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0e6fdb-15cb-40df-9ca1-8de846c135c0 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:09.769793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c52e7-de99-4d62-97be-f20a6ddb4d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:09.770403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c52e7-de99-4d62-97be-f20a6ddb4d06 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:09.770680 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c52e7-de99-4d62-97be-f20a6ddb4d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:09.770951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c52e7-de99-4d62-97be-f20a6ddb4d06 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:09.771654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c52e7-de99-4d62-97be-f20a6ddb4d06 None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:09.771961 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c52e7-de99-4d62-97be-f20a6ddb4d06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:09.773386 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c52e7-de99-4d62-97be-f20a6ddb4d06 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:09.781779 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 325/645] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:55:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:09.782562 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c52e7-de99-4d62-97be-f20a6ddb4d06 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:09.789815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558d5e9-d527-47f8-9779-baaaad199d61 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:09.824652 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87dd951-0964-4df3-ac5c-e73e6e161103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:09.825265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87dd951-0964-4df3-ac5c-e73e6e161103 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:09.825537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87dd951-0964-4df3-ac5c-e73e6e161103 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:09.825808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87dd951-0964-4df3-ac5c-e73e6e161103 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:09.826401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:09.826513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87dd951-0964-4df3-ac5c-e73e6e161103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:09.826725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87dd951-0964-4df3-ac5c-e73e6e161103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:09.828117 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87dd951-0964-4df3-ac5c-e73e6e161103 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:09.842091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 321/646] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:09.842934 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87dd951-0964-4df3-ac5c-e73e6e161103 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:09.855593 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:09.899830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c52e7-de99-4d62-97be-f20a6ddb4d06 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:09.934018 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f26a0-27c0-4fbb-a3d2-c74871439aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:09.934732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f26a0-27c0-4fbb-a3d2-c74871439aba None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:09.935011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f26a0-27c0-4fbb-a3d2-c74871439aba None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:09.935387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1f26a0-27c0-4fbb-a3d2-c74871439aba None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:09.936147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f26a0-27c0-4fbb-a3d2-c74871439aba None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:09.936448 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f26a0-27c0-4fbb-a3d2-c74871439aba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:09.938060 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1f26a0-27c0-4fbb-a3d2-c74871439aba None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:09.943840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87dd951-0964-4df3-ac5c-e73e6e161103 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:09.957213 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 326/647] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Sep 8 15:55:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:09.962909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f26a0-27c0-4fbb-a3d2-c74871439aba None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:09.984912 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82592201-7f4e-4bdd-856c-62958785f189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:09.985714 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82592201-7f4e-4bdd-856c-62958785f189 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:09.986041 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82592201-7f4e-4bdd-856c-62958785f189 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:09.986316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82592201-7f4e-4bdd-856c-62958785f189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:09.997402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82592201-7f4e-4bdd-856c-62958785f189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:09.998893 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82592201-7f4e-4bdd-856c-62958785f189 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-82592201-7f4e-4bdd-856c-62958785f189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:10.013263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:10.018886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:10.020080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:10.020080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:10.020080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:10.020080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 322/648] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:10.020080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82592201-7f4e-4bdd-856c-62958785f189 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:10.069799 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:10.077929 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1f26a0-27c0-4fbb-a3d2-c74871439aba None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:10.078073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:10.116514 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fc431-d235-43dc-b87f-7338b703a2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:10.117063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fc431-d235-43dc-b87f-7338b703a2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:10.117344 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fc431-d235-43dc-b87f-7338b703a2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:10.117619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fc431-d235-43dc-b87f-7338b703a2b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:10.118813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82592201-7f4e-4bdd-856c-62958785f189 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:10.129913 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fc431-d235-43dc-b87f-7338b703a2b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:10.131355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fc431-d235-43dc-b87f-7338b703a2b9 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-b66fc431-d235-43dc-b87f-7338b703a2b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:10.145863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:10.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:10.147730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:10.147730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:10.147730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:10.147730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 327/649] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:10.147953 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fc431-d235-43dc-b87f-7338b703a2b9 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:10.158557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4b0ee-eeef-4fa0-92ff-7f84755b9398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:10.159192 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4b0ee-eeef-4fa0-92ff-7f84755b9398 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:10.159494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4b0ee-eeef-4fa0-92ff-7f84755b9398 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:10.159762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4b0ee-eeef-4fa0-92ff-7f84755b9398 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:10.170058 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4b0ee-eeef-4fa0-92ff-7f84755b9398 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:10.171394 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4b0ee-eeef-4fa0-92ff-7f84755b9398 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-92a4b0ee-eeef-4fa0-92ff-7f84755b9398 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:10.185049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:10.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:10.186970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:10.186970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:10.186970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:10.186970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:10.186970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:10.186970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:10.186970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:10.186970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:10.186970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 15:55:10.186970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:10.191477 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 323/650] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:55:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:10.192516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4b0ee-eeef-4fa0-92ff-7f84755b9398 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:10.263609 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fc431-d235-43dc-b87f-7338b703a2b9 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:10.302394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650228ec-05a7-4a81-ad06-2699103cdb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:10.303364 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650228ec-05a7-4a81-ad06-2699103cdb24 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:10.303672 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650228ec-05a7-4a81-ad06-2699103cdb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:10.303947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650228ec-05a7-4a81-ad06-2699103cdb24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:10.304887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4b0ee-eeef-4fa0-92ff-7f84755b9398 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:10.324714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:10.329620 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:10.342748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9584c6-a241-4c31-82cd-e9972d1129fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:10.343379 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9584c6-a241-4c31-82cd-e9972d1129fc None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:10.343694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9584c6-a241-4c31-82cd-e9972d1129fc None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:10.343965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9584c6-a241-4c31-82cd-e9972d1129fc None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:10.344613 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9584c6-a241-4c31-82cd-e9972d1129fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:10.344913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9584c6-a241-4c31-82cd-e9972d1129fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:10.346421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9584c6-a241-4c31-82cd-e9972d1129fc None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:10.359112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 324/651] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:10.364346 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9584c6-a241-4c31-82cd-e9972d1129fc None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:10.377106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650228ec-05a7-4a81-ad06-2699103cdb24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:10.378608 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650228ec-05a7-4a81-ad06-2699103cdb24 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:10.459889 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9584c6-a241-4c31-82cd-e9972d1129fc None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:10.473622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 328/652] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:55:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:55:10.474459 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650228ec-05a7-4a81-ad06-2699103cdb24 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:10.493399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc076e8-91da-4829-a3b8-7b73daa67c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:10.494001 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc076e8-91da-4829-a3b8-7b73daa67c54 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:10.494269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc076e8-91da-4829-a3b8-7b73daa67c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:10.494530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc076e8-91da-4829-a3b8-7b73daa67c54 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:10.495165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc076e8-91da-4829-a3b8-7b73daa67c54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:10.495483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc076e8-91da-4829-a3b8-7b73daa67c54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:10.496854 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc076e8-91da-4829-a3b8-7b73daa67c54 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:10.508098 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 325/653] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:10.508862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc076e8-91da-4829-a3b8-7b73daa67c54 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:10.527843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:10.550518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:10.579768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650228ec-05a7-4a81-ad06-2699103cdb24 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:10.607789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc076e8-91da-4829-a3b8-7b73daa67c54 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:10.613404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af87c70-ac9f-47d2-90f6-b3241da74579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:10.614010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af87c70-ac9f-47d2-90f6-b3241da74579 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:10.614288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af87c70-ac9f-47d2-90f6-b3241da74579 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:10.614552 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af87c70-ac9f-47d2-90f6-b3241da74579 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:10.615277 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af87c70-ac9f-47d2-90f6-b3241da74579 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:10.615610 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af87c70-ac9f-47d2-90f6-b3241da74579 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:10.617044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af87c70-ac9f-47d2-90f6-b3241da74579 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:10.633531 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 329/654] 10.222.0.27 () {56 vars in 1137 bytes} [Thu Sep 8 15:55:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:10.634624 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dadb0a-2b05-4e49-9c08-3c2672bf241d None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:10.634904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dadb0a-2b05-4e49-9c08-3c2672bf241d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:10.635184 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dadb0a-2b05-4e49-9c08-3c2672bf241d None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:10.639494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 330/655] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:10.639494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dadb0a-2b05-4e49-9c08-3c2672bf241d None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:10.644880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe04ad-0a2b-4f3b-821c-673c6f9d21e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:10.645914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe04ad-0a2b-4f3b-821c-673c6f9d21e7 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:10.646195 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe04ad-0a2b-4f3b-821c-673c6f9d21e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:10.646464 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fe04ad-0a2b-4f3b-821c-673c6f9d21e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:10.653679 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe04ad-0a2b-4f3b-821c-673c6f9d21e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:10.655122 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fe04ad-0a2b-4f3b-821c-673c6f9d21e7 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:10.666200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-49fe04ad-0a2b-4f3b-821c-673c6f9d21e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:10.666200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:10.666200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:10.666200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:10.666200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:10.666200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:10.666200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:10.666200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:10.666200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:10.666200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:10.666200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:10.666200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:10.666200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:10.666200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:10.666200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:10.667113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:10.668435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:10.668435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:10.668435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:10.668435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:10.668435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:10.668435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:10.668435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:10.668435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:10.668435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 326/656] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:10.668435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe04ad-0a2b-4f3b-821c-673c6f9d21e7 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:10.738927 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dadb0a-2b05-4e49-9c08-3c2672bf241d None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:10.746418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:10.765062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fe04ad-0a2b-4f3b-821c-673c6f9d21e7 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:10.766555 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:10.774737 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0662a-8f00-4473-a54b-14234e28771d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:10.775370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0662a-8f00-4473-a54b-14234e28771d None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:10.775677 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0662a-8f00-4473-a54b-14234e28771d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:10.775941 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0662a-8f00-4473-a54b-14234e28771d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:10.786206 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d0662a-8f00-4473-a54b-14234e28771d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:10.787686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d0662a-8f00-4473-a54b-14234e28771d None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:10.798670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6dc4e3-c09b-4461-a8cc-1318c05789b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:10.799372 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dc4e3-c09b-4461-a8cc-1318c05789b5 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:10.799687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dc4e3-c09b-4461-a8cc-1318c05789b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:10.799933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6dc4e3-c09b-4461-a8cc-1318c05789b5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-83d0662a-8f00-4473-a54b-14234e28771d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:10.802182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:10.803242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:10.804196 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:10.804196 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:10.804196 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:10.804196 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 331/657] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:10.804431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0662a-8f00-4473-a54b-14234e28771d None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:10.808083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dc4e3-c09b-4461-a8cc-1318c05789b5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:10.809542 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6dc4e3-c09b-4461-a8cc-1318c05789b5 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-cc6dc4e3-c09b-4461-a8cc-1318c05789b5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:10.820635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:10.821758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:10.822702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:10.822702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:10.822702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:10.822702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:10.822702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:10.822702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:10.822702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:10.822702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:10.822702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 15:55:10.822702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:10.822702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 327/658] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:55:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:10.823341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6a69b-a416-4666-9db8-5591d5d1826f None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:10.823410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6a69b-a416-4666-9db8-5591d5d1826f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:10.823710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6a69b-a416-4666-9db8-5591d5d1826f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:10.841614 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-e2e6a69b-a416-4666-9db8-5591d5d1826f None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:10.842790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e2e6a69b-a416-4666-9db8-5591d5d1826f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:10.906933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 328/659] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:55:10.907708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6a69b-a416-4666-9db8-5591d5d1826f None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:10.923663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0662a-8f00-4473-a54b-14234e28771d None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:10.959477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25355836-c735-4931-8007-4333ea23e514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:10.960248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25355836-c735-4931-8007-4333ea23e514 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:10.960617 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25355836-c735-4931-8007-4333ea23e514 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:10.960990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25355836-c735-4931-8007-4333ea23e514 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:10.991231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:11.001040 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:11.011171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6a69b-a416-4666-9db8-5591d5d1826f None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:11.038436 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25355836-c735-4931-8007-4333ea23e514 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:11.040165 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25355836-c735-4931-8007-4333ea23e514 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:11.054066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d51bd3-0b75-4102-99ed-8c1c3175c64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:11.054675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d51bd3-0b75-4102-99ed-8c1c3175c64a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:11.054970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d51bd3-0b75-4102-99ed-8c1c3175c64a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:11.055472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d51bd3-0b75-4102-99ed-8c1c3175c64a None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:11.056616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d51bd3-0b75-4102-99ed-8c1c3175c64a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:11.056919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d51bd3-0b75-4102-99ed-8c1c3175c64a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:11.058854 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d51bd3-0b75-4102-99ed-8c1c3175c64a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:11.073119 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 329/660] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:11.073902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d51bd3-0b75-4102-99ed-8c1c3175c64a None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:11.153425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 332/661] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:55:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:55:11.154783 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25355836-c735-4931-8007-4333ea23e514 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:11.178321 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d51bd3-0b75-4102-99ed-8c1c3175c64a None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:11.210992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832e85d-f73a-4d52-a137-1752a9cc2020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:11.211633 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832e85d-f73a-4d52-a137-1752a9cc2020 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:11.212270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832e85d-f73a-4d52-a137-1752a9cc2020 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:11.212532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3832e85d-f73a-4d52-a137-1752a9cc2020 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:11.213165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832e85d-f73a-4d52-a137-1752a9cc2020 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:11.213460 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832e85d-f73a-4d52-a137-1752a9cc2020 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:11.214211 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:11.214822 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3832e85d-f73a-4d52-a137-1752a9cc2020 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:11.226469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 330/662] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:11.227272 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832e85d-f73a-4d52-a137-1752a9cc2020 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:11.241160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:11.256899 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25355836-c735-4931-8007-4333ea23e514 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:11.288845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f046c8e-674d-43c4-8ddd-d441eca0e4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:11.289442 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f046c8e-674d-43c4-8ddd-d441eca0e4fb None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:11.289713 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f046c8e-674d-43c4-8ddd-d441eca0e4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:11.289976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f046c8e-674d-43c4-8ddd-d441eca0e4fb None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:11.290698 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f046c8e-674d-43c4-8ddd-d441eca0e4fb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:11.290993 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f046c8e-674d-43c4-8ddd-d441eca0e4fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:11.292438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f046c8e-674d-43c4-8ddd-d441eca0e4fb None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:11.308647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 333/663] 10.222.0.27 () {56 vars in 1139 bytes} [Thu Sep 8 15:55:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:11.309437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f046c8e-674d-43c4-8ddd-d441eca0e4fb None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:11.331020 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3832e85d-f73a-4d52-a137-1752a9cc2020 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:11.366471 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2994f9d-4531-48bb-a1ec-f7bec2f324b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:11.367081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2994f9d-4531-48bb-a1ec-f7bec2f324b3 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:11.367350 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2994f9d-4531-48bb-a1ec-f7bec2f324b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:11.367648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2994f9d-4531-48bb-a1ec-f7bec2f324b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:11.368375 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2994f9d-4531-48bb-a1ec-f7bec2f324b3 None admin] RBAC: Adding query filter params (role.id=b256252b45e541a491376a134f502183, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=89df988d73ff48f3b7a92d335df9b633) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:11.368676 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2994f9d-4531-48bb-a1ec-f7bec2f324b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:11.370112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2994f9d-4531-48bb-a1ec-f7bec2f324b3 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:11.375718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 331/664] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:55:11 2022] GET /identity/v3/role_assignments?user.id=89df988d73ff48f3b7a92d335df9b633&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=b256252b45e541a491376a134f502183 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:11.376474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2994f9d-4531-48bb-a1ec-f7bec2f324b3 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:11.413403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f046c8e-674d-43c4-8ddd-d441eca0e4fb None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:11.432518 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:11.453045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c7cc9-ad29-437f-9bfe-2b3bb9d37976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:11.453936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c7cc9-ad29-437f-9bfe-2b3bb9d37976 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:11.454278 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c7cc9-ad29-437f-9bfe-2b3bb9d37976 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:11.454949 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c7cc9-ad29-437f-9bfe-2b3bb9d37976 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:11.467180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c7cc9-ad29-437f-9bfe-2b3bb9d37976 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:11.468783 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c7cc9-ad29-437f-9bfe-2b3bb9d37976 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:11.472944 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-0b5c7cc9-ad29-437f-9bfe-2b3bb9d37976 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:11.486988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:11.492959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:11.494065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:11.494065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:11.494065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:11.494065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 334/665] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:11.494065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c7cc9-ad29-437f-9bfe-2b3bb9d37976 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:11.512087 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2994f9d-4531-48bb-a1ec-f7bec2f324b3 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:11.551489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220fda55-02d8-4a51-ad7b-f606feb3304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:11.552175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fda55-02d8-4a51-ad7b-f606feb3304f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:11.552449 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fda55-02d8-4a51-ad7b-f606feb3304f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:11.552720 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220fda55-02d8-4a51-ad7b-f606feb3304f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:11.564920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220fda55-02d8-4a51-ad7b-f606feb3304f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-220fda55-02d8-4a51-ad7b-f606feb3304f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:11.570322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:11.571500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:11.572560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:11.572560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:11.572560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:11.572560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:11.572560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:11.572560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:11.572560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:11.572560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:11.572560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:11.572560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:11.572560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:11.572560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:11.574963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 332/666] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:55:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:11.575354 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220fda55-02d8-4a51-ad7b-f606feb3304f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:11.605031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c7cc9-ad29-437f-9bfe-2b3bb9d37976 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:11.640415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa4e9e-f7b6-4a23-9b4a-f8438b7bf6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:11.641197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa4e9e-f7b6-4a23-9b4a-f8438b7bf6e5 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:11.641542 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa4e9e-f7b6-4a23-9b4a-f8438b7bf6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:11.641809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa4e9e-f7b6-4a23-9b4a-f8438b7bf6e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:11.648512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa4e9e-f7b6-4a23-9b4a-f8438b7bf6e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:11.649966 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa4e9e-f7b6-4a23-9b4a-f8438b7bf6e5 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:11.662579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-30aa4e9e-f7b6-4a23-9b4a-f8438b7bf6e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:11.662579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:11.662579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:11.662579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:11.662579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:11.662579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:11.662579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:11.662579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:11.662579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:11.662579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:11.662579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:11.662579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:11.662579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:11.662579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:11.662579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:11.663754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:11.664878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:11.664878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:11.664878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:11.664878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:11.664878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:11.664878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:11.664878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:11.664878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:11.668415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 335/667] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:11.669171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa4e9e-f7b6-4a23-9b4a-f8438b7bf6e5 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:11.685881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220fda55-02d8-4a51-ad7b-f606feb3304f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:11.698708 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:11.722640 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acaa8b-271f-4864-aac6-812de5af867c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:11.723242 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acaa8b-271f-4864-aac6-812de5af867c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:11.723531 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acaa8b-271f-4864-aac6-812de5af867c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:11.723797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acaa8b-271f-4864-aac6-812de5af867c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:11.727182 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:11.776080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa4e9e-f7b6-4a23-9b4a-f8438b7bf6e5 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:11.794239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acaa8b-271f-4864-aac6-812de5af867c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:11.795748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acaa8b-271f-4864-aac6-812de5af867c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:11.816484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3466f9-6174-4c33-9e25-451fbcaa40d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:11.817522 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3466f9-6174-4c33-9e25-451fbcaa40d0 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:11.817802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3466f9-6174-4c33-9e25-451fbcaa40d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:11.818070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3466f9-6174-4c33-9e25-451fbcaa40d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:11.818978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3466f9-6174-4c33-9e25-451fbcaa40d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:11.819279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3466f9-6174-4c33-9e25-451fbcaa40d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:11.820700 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3466f9-6174-4c33-9e25-451fbcaa40d0 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:11.839094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 336/668] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:11.839094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3466f9-6174-4c33-9e25-451fbcaa40d0 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:11.900472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 333/669] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:55:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:55:11.900472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acaa8b-271f-4864-aac6-812de5af867c None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:11.935637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:11.937487 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:11.951237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3466f9-6174-4c33-9e25-451fbcaa40d0 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:11.990336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c67b9-b42c-490a-a56c-a0fcb962be13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:11.990956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c67b9-b42c-490a-a56c-a0fcb962be13 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:11.991225 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c67b9-b42c-490a-a56c-a0fcb962be13 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:11.991523 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c67b9-b42c-490a-a56c-a0fcb962be13 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:11.992201 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c67b9-b42c-490a-a56c-a0fcb962be13 None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:11.992503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c67b9-b42c-490a-a56c-a0fcb962be13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:11.993882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c67b9-b42c-490a-a56c-a0fcb962be13 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:12.002493 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 337/670] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:55:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:12.004537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c67b9-b42c-490a-a56c-a0fcb962be13 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:12.005776 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acaa8b-271f-4864-aac6-812de5af867c None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:12.040919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41c1d3-8ef8-4af7-9cf9-510d77c26c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:12.041525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41c1d3-8ef8-4af7-9cf9-510d77c26c74 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:12.041796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41c1d3-8ef8-4af7-9cf9-510d77c26c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:12.042065 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41c1d3-8ef8-4af7-9cf9-510d77c26c74 None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:12.043531 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41c1d3-8ef8-4af7-9cf9-510d77c26c74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:12.048017 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41c1d3-8ef8-4af7-9cf9-510d77c26c74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:12.049927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41c1d3-8ef8-4af7-9cf9-510d77c26c74 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:12.062716 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 334/671] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:12.063831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41c1d3-8ef8-4af7-9cf9-510d77c26c74 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:12.120007 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c67b9-b42c-490a-a56c-a0fcb962be13 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:12.158531 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137816e-aa69-425c-b5ef-1f1208a1a6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:12.159174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137816e-aa69-425c-b5ef-1f1208a1a6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:12.159470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137816e-aa69-425c-b5ef-1f1208a1a6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:12.159749 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4137816e-aa69-425c-b5ef-1f1208a1a6c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:12.164322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:12.168786 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137816e-aa69-425c-b5ef-1f1208a1a6c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:12.168786 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4137816e-aa69-425c-b5ef-1f1208a1a6c3 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:12.175275 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41c1d3-8ef8-4af7-9cf9-510d77c26c74 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:12.185811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-4137816e-aa69-425c-b5ef-1f1208a1a6c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:12.185811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:12.185811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:12.185811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:12.185811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:12.185811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:12.185811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:12.185811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:12.185811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:12.185811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:12.185811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:12.185811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:12.185811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:12.185811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:12.185811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:12.186945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:12.188218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:12.188218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:12.188218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:12.188218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:12.188218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:12.188218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:12.188218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:12.188218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:12.188218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 338/672] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:12.188218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137816e-aa69-425c-b5ef-1f1208a1a6c3 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:12.197684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:12.221496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5adba-e973-420c-8057-774ed164e17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:12.222122 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5adba-e973-420c-8057-774ed164e17f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:12.222393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5adba-e973-420c-8057-774ed164e17f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:12.222661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5adba-e973-420c-8057-774ed164e17f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:12.233367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c5adba-e973-420c-8057-774ed164e17f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:12.234806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c5adba-e973-420c-8057-774ed164e17f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-b1c5adba-e973-420c-8057-774ed164e17f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:12.249702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:12.250876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:12.252003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:12.252003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:12.252003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:12.255481 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 335/673] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:12.256323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5adba-e973-420c-8057-774ed164e17f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:12.333336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4137816e-aa69-425c-b5ef-1f1208a1a6c3 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:12.392381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd28e52-1a05-47bb-b7cd-2959591d7de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:12.393101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd28e52-1a05-47bb-b7cd-2959591d7de1 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:12.393758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd28e52-1a05-47bb-b7cd-2959591d7de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:12.394033 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd28e52-1a05-47bb-b7cd-2959591d7de1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:12.407708 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd28e52-1a05-47bb-b7cd-2959591d7de1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:12.409797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd28e52-1a05-47bb-b7cd-2959591d7de1 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:12.413179 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c5adba-e973-420c-8057-774ed164e17f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-bdd28e52-1a05-47bb-b7cd-2959591d7de1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:12.427289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:12.428640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:12.429584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:12.429584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:12.429584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:12.431605 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 339/674] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:12.432510 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd28e52-1a05-47bb-b7cd-2959591d7de1 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:12.465353 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffef05-3c8b-44b0-a1b3-29e9766ca24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:12.466218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffef05-3c8b-44b0-a1b3-29e9766ca24c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:12.466494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffef05-3c8b-44b0-a1b3-29e9766ca24c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:12.466762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffef05-3c8b-44b0-a1b3-29e9766ca24c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:12.467511 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffef05-3c8b-44b0-a1b3-29e9766ca24c None admin] RBAC: Adding query filter params (role.id=b256252b45e541a491376a134f502183, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=6e9209e77ce44b128e25a8ddcf78f8b5) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:12.467821 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffef05-3c8b-44b0-a1b3-29e9766ca24c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:12.469384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffef05-3c8b-44b0-a1b3-29e9766ca24c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:12.476934 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 336/675] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:55:12 2022] GET /identity/v3/role_assignments?user.id=6e9209e77ce44b128e25a8ddcf78f8b5&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=b256252b45e541a491376a134f502183 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:12.477571 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffef05-3c8b-44b0-a1b3-29e9766ca24c None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:12.494199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:12.540375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:12.612109 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd28e52-1a05-47bb-b7cd-2959591d7de1 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:12.656235 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffef05-3c8b-44b0-a1b3-29e9766ca24c None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:12.660097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7bd2b-ead3-43ff-97c7-3949b7999196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:12.661049 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7bd2b-ead3-43ff-97c7-3949b7999196 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:12.661322 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7bd2b-ead3-43ff-97c7-3949b7999196 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:12.661589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7bd2b-ead3-43ff-97c7-3949b7999196 None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:12.662318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7bd2b-ead3-43ff-97c7-3949b7999196 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:12.662622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7bd2b-ead3-43ff-97c7-3949b7999196 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:12.664122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b7bd2b-ead3-43ff-97c7-3949b7999196 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:12.682795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 340/676] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:12.682795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7bd2b-ead3-43ff-97c7-3949b7999196 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:12.717035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70a085-087e-4602-8517-8a52c118fc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:12.718026 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70a085-087e-4602-8517-8a52c118fc8a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:12.718371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70a085-087e-4602-8517-8a52c118fc8a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:12.718716 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70a085-087e-4602-8517-8a52c118fc8a None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:12.719935 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70a085-087e-4602-8517-8a52c118fc8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:12.719935 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70a085-087e-4602-8517-8a52c118fc8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:12.721372 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70a085-087e-4602-8517-8a52c118fc8a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:12.734793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 337/677] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:12.735750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70a085-087e-4602-8517-8a52c118fc8a None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:12.813509 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7bd2b-ead3-43ff-97c7-3949b7999196 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:12.820260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:12.841173 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:12.858674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8882be9-27d8-4eb6-b777-63736439e667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:12.859285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8882be9-27d8-4eb6-b777-63736439e667 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:12.859602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8882be9-27d8-4eb6-b777-63736439e667 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:12.859842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70a085-087e-4602-8517-8a52c118fc8a None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:12.859961 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8882be9-27d8-4eb6-b777-63736439e667 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:12.860506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8882be9-27d8-4eb6-b777-63736439e667 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:12.860803 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8882be9-27d8-4eb6-b777-63736439e667 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:12.862294 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8882be9-27d8-4eb6-b777-63736439e667 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:12.874161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 341/678] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:12.875002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8882be9-27d8-4eb6-b777-63736439e667 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:12.908912 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249904b-a77e-4ee3-94ce-2310aedbf015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:12.909579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249904b-a77e-4ee3-94ce-2310aedbf015 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:12.909855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249904b-a77e-4ee3-94ce-2310aedbf015 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:12.910366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249904b-a77e-4ee3-94ce-2310aedbf015 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:12.911508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249904b-a77e-4ee3-94ce-2310aedbf015 None admin] RBAC: Adding query filter params (role.id=b256252b45e541a491376a134f502183, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=f24ee16c043244938901c4e850dea967) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:12.912065 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249904b-a77e-4ee3-94ce-2310aedbf015 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:12.913521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249904b-a77e-4ee3-94ce-2310aedbf015 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:12.925989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 338/679] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:55:12 2022] GET /identity/v3/role_assignments?user.id=f24ee16c043244938901c4e850dea967&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=b256252b45e541a491376a134f502183 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:12.927945 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249904b-a77e-4ee3-94ce-2310aedbf015 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:13.050946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8882be9-27d8-4eb6-b777-63736439e667 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:13.090730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6275f-859c-44f9-acac-410fed1ab1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:13.091394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6275f-859c-44f9-acac-410fed1ab1ef None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:13.091695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6275f-859c-44f9-acac-410fed1ab1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:13.091969 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6275f-859c-44f9-acac-410fed1ab1ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:13.092346 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249904b-a77e-4ee3-94ce-2310aedbf015 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:13.108959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6275f-859c-44f9-acac-410fed1ab1ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:13.110714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6275f-859c-44f9-acac-410fed1ab1ef None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:13.133556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-abb6275f-859c-44f9-acac-410fed1ab1ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:13.133556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:13.133556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:13.133556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:13.133556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:13.133556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:13.133556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:13.133556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:13.133556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:13.133556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:13.133556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:13.133556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:13.133556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:13.133556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:13.133556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:13.134709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:13.135791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:13.135791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:13.135791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:13.135791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:13.135791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:13.135791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:13.135791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:13.135791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:13.147537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 342/680] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:13.154705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6275f-859c-44f9-acac-410fed1ab1ef None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:13.170912 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da213010-12e6-424d-b496-1786880c90e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:13.171587 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da213010-12e6-424d-b496-1786880c90e9 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:13.171857 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da213010-12e6-424d-b496-1786880c90e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:13.172120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da213010-12e6-424d-b496-1786880c90e9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:13.179807 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:13.181454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da213010-12e6-424d-b496-1786880c90e9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:13.183221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da213010-12e6-424d-b496-1786880c90e9 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-da213010-12e6-424d-b496-1786880c90e9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:13.196651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:13.197951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:13.199101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:13.199101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:13.199101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:13.199101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:13.199101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:13.199101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:13.199101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:13.199101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:13.199101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 15:55:13.199101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:13.199101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 339/681] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:55:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:13.204591 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da213010-12e6-424d-b496-1786880c90e9 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:13.229842 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:13.299919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6275f-859c-44f9-acac-410fed1ab1ef None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:13.342564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da213010-12e6-424d-b496-1786880c90e9 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:13.348088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65958fc-3bea-409c-aa71-aac42338d254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:13.349022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65958fc-3bea-409c-aa71-aac42338d254 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:13.349517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65958fc-3bea-409c-aa71-aac42338d254 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:13.350066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65958fc-3bea-409c-aa71-aac42338d254 None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:13.350971 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65958fc-3bea-409c-aa71-aac42338d254 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:13.351522 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65958fc-3bea-409c-aa71-aac42338d254 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:13.353443 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65958fc-3bea-409c-aa71-aac42338d254 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:13.369032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 343/682] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:13.374742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65958fc-3bea-409c-aa71-aac42338d254 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:13.384479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaee8b-740d-4e5c-8394-5d04566bfc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:13.386238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaee8b-740d-4e5c-8394-5d04566bfc0b None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:13.386238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaee8b-740d-4e5c-8394-5d04566bfc0b None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:13.386238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaee8b-740d-4e5c-8394-5d04566bfc0b None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:13.386804 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abaee8b-740d-4e5c-8394-5d04566bfc0b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:13.387107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abaee8b-740d-4e5c-8394-5d04566bfc0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:13.389042 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abaee8b-740d-4e5c-8394-5d04566bfc0b None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:13.412937 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 340/683] 10.222.0.27 () {56 vars in 1135 bytes} [Thu Sep 8 15:55:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:13.420384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaee8b-740d-4e5c-8394-5d04566bfc0b None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:13.486473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:13.498099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65958fc-3bea-409c-aa71-aac42338d254 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:13.499786 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:13.541917 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaee8b-740d-4e5c-8394-5d04566bfc0b None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:13.548342 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96a8257-8f47-4691-93bf-8d2f1f95622c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:13.548968 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a8257-8f47-4691-93bf-8d2f1f95622c None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:13.549365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a8257-8f47-4691-93bf-8d2f1f95622c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:13.549638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a8257-8f47-4691-93bf-8d2f1f95622c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:13.550431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a8257-8f47-4691-93bf-8d2f1f95622c None admin] RBAC: Adding query filter params (role.id=b256252b45e541a491376a134f502183, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=3aae65f9ec394f97aeafad43932dc49c) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:13.550751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a8257-8f47-4691-93bf-8d2f1f95622c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:13.552242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96a8257-8f47-4691-93bf-8d2f1f95622c None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:13.560833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 344/684] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:55:13 2022] GET /identity/v3/role_assignments?user.id=3aae65f9ec394f97aeafad43932dc49c&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=b256252b45e541a491376a134f502183 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:13.597524 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124f4c6-3946-4826-9bd5-91a61fb28b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:13.598516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124f4c6-3946-4826-9bd5-91a61fb28b46 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:13.599005 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124f4c6-3946-4826-9bd5-91a61fb28b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:13.599486 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124f4c6-3946-4826-9bd5-91a61fb28b46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:13.702643 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124f4c6-3946-4826-9bd5-91a61fb28b46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:13.704524 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124f4c6-3946-4826-9bd5-91a61fb28b46 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:13.752675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27693ec2-abaa-481f-b3ee-b1ad065eb878 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:13.753082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27693ec2-abaa-481f-b3ee-b1ad065eb878 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:13.753261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27693ec2-abaa-481f-b3ee-b1ad065eb878 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:13.755143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 345/685] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:13.763465 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190ec10-03a1-4d5f-9a28-9bc3627f1310 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:13.763465 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190ec10-03a1-4d5f-9a28-9bc3627f1310 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:13.763465 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190ec10-03a1-4d5f-9a28-9bc3627f1310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:13.774237 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:13.781379 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-c190ec10-03a1-4d5f-9a28-9bc3627f1310 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:13.783012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c190ec10-03a1-4d5f-9a28-9bc3627f1310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:13.858566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 341/686] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:55:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:55:13.863957 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124f4c6-3946-4826-9bd5-91a61fb28b46 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:13.914197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 346/687] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:55:13.998009 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124f4c6-3946-4826-9bd5-91a61fb28b46 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:14.035423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19b6c7-0b30-4f99-b0cc-8a8c1ebe6ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:14.036087 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19b6c7-0b30-4f99-b0cc-8a8c1ebe6ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:14.036420 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19b6c7-0b30-4f99-b0cc-8a8c1ebe6ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:14.036688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d19b6c7-0b30-4f99-b0cc-8a8c1ebe6ca1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:14.047882 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19b6c7-0b30-4f99-b0cc-8a8c1ebe6ca1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:14.049389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d19b6c7-0b30-4f99-b0cc-8a8c1ebe6ca1 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-6d19b6c7-0b30-4f99-b0cc-8a8c1ebe6ca1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:14.064396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:14.065472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:14.066374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:14.066374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:14.066374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:14.066374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 342/688] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:14.072664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190ec10-03a1-4d5f-9a28-9bc3627f1310 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:14.218425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:14.224595 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190ec10-03a1-4d5f-9a28-9bc3627f1310 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:14.272750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2c382-4ef9-4c6c-8a4f-32e26e506d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:14.273393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2c382-4ef9-4c6c-8a4f-32e26e506d87 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:14.273667 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2c382-4ef9-4c6c-8a4f-32e26e506d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:14.273934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e2c382-4ef9-4c6c-8a4f-32e26e506d87 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:14.274611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2c382-4ef9-4c6c-8a4f-32e26e506d87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:14.274968 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2c382-4ef9-4c6c-8a4f-32e26e506d87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:14.276460 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e2c382-4ef9-4c6c-8a4f-32e26e506d87 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:14.278800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19b6c7-0b30-4f99-b0cc-8a8c1ebe6ca1 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:14.291359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 347/689] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:14.297621 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2c382-4ef9-4c6c-8a4f-32e26e506d87 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:14.298427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:14.413496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e2c382-4ef9-4c6c-8a4f-32e26e506d87 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:14.471931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89eca30-7f17-4b8c-944e-223c63ea53fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:14.472488 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eca30-7f17-4b8c-944e-223c63ea53fe None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:14.472765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eca30-7f17-4b8c-944e-223c63ea53fe None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:14.473062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89eca30-7f17-4b8c-944e-223c63ea53fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:14.488037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89eca30-7f17-4b8c-944e-223c63ea53fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:14.489497 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89eca30-7f17-4b8c-944e-223c63ea53fe None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:14.508453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-f89eca30-7f17-4b8c-944e-223c63ea53fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:14.508453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:14.508453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:14.508453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:14.508453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:14.508453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:14.508453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:14.508453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:14.508453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:14.508453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:14.508453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:14.508453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:14.508453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:14.508453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:14.508453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:14.510151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:14.511268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:14.511268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:14.511268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:14.511268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:14.511268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:14.511268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:14.511268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:14.511268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:14.511268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 348/690] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:14.514900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d19b6c7-0b30-4f99-b0cc-8a8c1ebe6ca1 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:14.521613 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89eca30-7f17-4b8c-944e-223c63ea53fe None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:14.559203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:14.593571 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75574049-31fe-47fa-b31b-b58cd795589c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:14.594631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75574049-31fe-47fa-b31b-b58cd795589c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:14.595109 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75574049-31fe-47fa-b31b-b58cd795589c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:14.595617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75574049-31fe-47fa-b31b-b58cd795589c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:14.596443 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75574049-31fe-47fa-b31b-b58cd795589c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:14.598180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75574049-31fe-47fa-b31b-b58cd795589c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-75574049-31fe-47fa-b31b-b58cd795589c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:14.620796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:14.621877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:14.622820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:14.622820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:14.622820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:14.622820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:14.622820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:14.622820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:14.622820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:14.622820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:14.622820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:14.622820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:14.622820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:14.622820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:14.622820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 15:55:14.622820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:14.632706 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 343/691] 10.222.0.27 () {56 vars in 1129 bytes} [Thu Sep 8 15:55:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:14.638793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75574049-31fe-47fa-b31b-b58cd795589c None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:14.645227 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89eca30-7f17-4b8c-944e-223c63ea53fe None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:14.685905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43f417-f2ad-4653-941e-5b888276a80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:14.686539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43f417-f2ad-4653-941e-5b888276a80b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:14.686816 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43f417-f2ad-4653-941e-5b888276a80b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:14.687087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43f417-f2ad-4653-941e-5b888276a80b None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:14.691498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43f417-f2ad-4653-941e-5b888276a80b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:14.691498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43f417-f2ad-4653-941e-5b888276a80b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:14.691498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43f417-f2ad-4653-941e-5b888276a80b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:14.704005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 349/692] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:14.710658 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43f417-f2ad-4653-941e-5b888276a80b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:14.793961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:14.835784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:14.854248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75574049-31fe-47fa-b31b-b58cd795589c None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:14.880546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43f417-f2ad-4653-941e-5b888276a80b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:14.925435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374552a2-0189-4ad9-8a05-61dd530550f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:14.926054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374552a2-0189-4ad9-8a05-61dd530550f5 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:14.926322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374552a2-0189-4ad9-8a05-61dd530550f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:14.926584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374552a2-0189-4ad9-8a05-61dd530550f5 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:14.927223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374552a2-0189-4ad9-8a05-61dd530550f5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:14.927545 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374552a2-0189-4ad9-8a05-61dd530550f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:14.929854 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374552a2-0189-4ad9-8a05-61dd530550f5 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:14.938578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 344/693] 10.222.0.27 () {56 vars in 1124 bytes} [Thu Sep 8 15:55:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:14.943361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374552a2-0189-4ad9-8a05-61dd530550f5 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:14.956418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783bd66-a30f-4de4-b912-f7e65961fe30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:14.957056 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783bd66-a30f-4de4-b912-f7e65961fe30 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:14.957335 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783bd66-a30f-4de4-b912-f7e65961fe30 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:14.957612 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783bd66-a30f-4de4-b912-f7e65961fe30 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:14.958348 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783bd66-a30f-4de4-b912-f7e65961fe30 None admin] RBAC: Adding query filter params (role.id=b256252b45e541a491376a134f502183, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=de54687547a14ab89cc41a1fabaab400) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:14.958651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783bd66-a30f-4de4-b912-f7e65961fe30 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:14.960129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783bd66-a30f-4de4-b912-f7e65961fe30 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:14.968337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 350/694] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:55:14 2022] GET /identity/v3/role_assignments?user.id=de54687547a14ab89cc41a1fabaab400&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=b256252b45e541a491376a134f502183 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:15.091467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374552a2-0189-4ad9-8a05-61dd530550f5 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:15.129654 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c6b6e-7b10-49db-892e-baec5c1179b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:15.130284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c6b6e-7b10-49db-892e-baec5c1179b6 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:15.130551 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c6b6e-7b10-49db-892e-baec5c1179b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:15.130815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c6b6e-7b10-49db-892e-baec5c1179b6 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:15.131504 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c6b6e-7b10-49db-892e-baec5c1179b6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:15.131808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c6b6e-7b10-49db-892e-baec5c1179b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:15.133255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c6b6e-7b10-49db-892e-baec5c1179b6 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:15.134887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:15.140982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 345/695] 10.222.0.27 () {56 vars in 1124 bytes} [Thu Sep 8 15:55:14 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:15.141180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d1db9-314b-46ed-886e-771e32b29a49 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:15.141324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d1db9-314b-46ed-886e-771e32b29a49 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:15.141594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d1db9-314b-46ed-886e-771e32b29a49 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:15.143218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 351/696] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:15.151521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b367cbd-e00c-40bb-8c98-adf4ffba9bf7 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:15.151521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b367cbd-e00c-40bb-8c98-adf4ffba9bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:15.151521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b367cbd-e00c-40bb-8c98-adf4ffba9bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:15.177287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-3b367cbd-e00c-40bb-8c98-adf4ffba9bf7 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:15.184299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3b367cbd-e00c-40bb-8c98-adf4ffba9bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:15.288300 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 346/697] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:55:15.568640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d1db9-314b-46ed-886e-771e32b29a49 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:15.684391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d1db9-314b-46ed-886e-771e32b29a49 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:15.692777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6f42a-01bd-4fa8-8e28-76f9ba4ed939 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:15.693298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6f42a-01bd-4fa8-8e28-76f9ba4ed939 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:15.693298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6f42a-01bd-4fa8-8e28-76f9ba4ed939 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:15.694879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 347/698] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:15.701014 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582534b-4b08-4cb8-98ad-afd9e5711bef None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:15.701014 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582534b-4b08-4cb8-98ad-afd9e5711bef None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:15.701014 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5582534b-4b08-4cb8-98ad-afd9e5711bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:15.726927 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c78883-246a-4c03-8d1c-9652b9c03ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:15.727621 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c78883-246a-4c03-8d1c-9652b9c03ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:15.727897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c78883-246a-4c03-8d1c-9652b9c03ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:15.728164 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c78883-246a-4c03-8d1c-9652b9c03ce4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:15.746666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:15.750383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-5582534b-4b08-4cb8-98ad-afd9e5711bef None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:15.751589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5582534b-4b08-4cb8-98ad-afd9e5711bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:15.754405 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c78883-246a-4c03-8d1c-9652b9c03ce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-c1c78883-246a-4c03-8d1c-9652b9c03ce4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:15.765946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:15.767008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:15.767972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:15.767972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:15.767972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:15.767972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:15.767972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:15.767972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:15.767972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:15.767972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:15.767972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:15.767972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:15.767972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:55:15.767972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:15.768627 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 352/699] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:55:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:15.782527 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c78883-246a-4c03-8d1c-9652b9c03ce4 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:15.882513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 348/700] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:55:15.911729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c78883-246a-4c03-8d1c-9652b9c03ce4 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:15.953548 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2e56f-3e69-4aca-9e28-e037d0dfd881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:15.954170 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2e56f-3e69-4aca-9e28-e037d0dfd881 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:15.954447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2e56f-3e69-4aca-9e28-e037d0dfd881 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:15.954717 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2e56f-3e69-4aca-9e28-e037d0dfd881 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:15.955487 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2e56f-3e69-4aca-9e28-e037d0dfd881 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:15.955793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2e56f-3e69-4aca-9e28-e037d0dfd881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:15.957169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2e56f-3e69-4aca-9e28-e037d0dfd881 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:15.967005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 353/701] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:55:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:15.972432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582534b-4b08-4cb8-98ad-afd9e5711bef None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:16.060157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:16.112075 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5582534b-4b08-4cb8-98ad-afd9e5711bef None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:16.147580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f247533-1b8a-4d33-a210-ed502b0bde98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:16.148116 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f247533-1b8a-4d33-a210-ed502b0bde98 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:16.148384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f247533-1b8a-4d33-a210-ed502b0bde98 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:16.148647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f247533-1b8a-4d33-a210-ed502b0bde98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:16.159224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f247533-1b8a-4d33-a210-ed502b0bde98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:16.160721 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f247533-1b8a-4d33-a210-ed502b0bde98 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-0f247533-1b8a-4d33-a210-ed502b0bde98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:16.179036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:16.180093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:16.181018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:16.181018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:16.181018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:16.181018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 349/702] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:16.192655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2e56f-3e69-4aca-9e28-e037d0dfd881 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:16.283215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:16.294343 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f247533-1b8a-4d33-a210-ed502b0bde98 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:16.347817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2e56f-3e69-4aca-9e28-e037d0dfd881 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:16.384949 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5e97b-4474-4dd1-9623-9155684c2b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:16.386128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5e97b-4474-4dd1-9623-9155684c2b61 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:16.386403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5e97b-4474-4dd1-9623-9155684c2b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:16.386673 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5e97b-4474-4dd1-9623-9155684c2b61 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:16.387626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5e97b-4474-4dd1-9623-9155684c2b61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:16.388025 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5e97b-4474-4dd1-9623-9155684c2b61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:16.390547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5e97b-4474-4dd1-9623-9155684c2b61 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:16.406731 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 354/703] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:16.408040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667cc77-655d-4c46-a2de-496275ae516c None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:16.408318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667cc77-655d-4c46-a2de-496275ae516c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:16.408738 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e667cc77-655d-4c46-a2de-496275ae516c None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:16.411578 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 355/704] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:16.412483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667cc77-655d-4c46-a2de-496275ae516c None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:16.426274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f247533-1b8a-4d33-a210-ed502b0bde98 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:16.437542 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:16.463930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53f2c9-6748-4f7b-8de2-f37ab2d4a109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:16.464701 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53f2c9-6748-4f7b-8de2-f37ab2d4a109 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:16.465035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53f2c9-6748-4f7b-8de2-f37ab2d4a109 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:16.465359 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53f2c9-6748-4f7b-8de2-f37ab2d4a109 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:16.477022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53f2c9-6748-4f7b-8de2-f37ab2d4a109 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:16.478632 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53f2c9-6748-4f7b-8de2-f37ab2d4a109 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-ce53f2c9-6748-4f7b-8de2-f37ab2d4a109 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:16.493472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:16.495322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:16.497452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:16.497452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:16.497452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:16.497452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 350/705] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:16.497452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b2875-8298-49ca-be73-a9c72889e125 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:16.497452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b2875-8298-49ca-be73-a9c72889e125 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:16.497452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b2875-8298-49ca-be73-a9c72889e125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:16.521264 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f70b2875-8298-49ca-be73-a9c72889e125 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:16.522434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f70b2875-8298-49ca-be73-a9c72889e125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:16.542328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e667cc77-655d-4c46-a2de-496275ae516c None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:16.570890 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:16.585386 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe3989-17ef-44eb-9c30-6070b02b650e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:16.586174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe3989-17ef-44eb-9c30-6070b02b650e None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:16.586566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe3989-17ef-44eb-9c30-6070b02b650e None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:16.587206 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe3989-17ef-44eb-9c30-6070b02b650e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:16.595568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe3989-17ef-44eb-9c30-6070b02b650e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:16.597264 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe3989-17ef-44eb-9c30-6070b02b650e None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:16.606001 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 351/706] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:55:16.606935 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b2875-8298-49ca-be73-a9c72889e125 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-dcfe3989-17ef-44eb-9c30-6070b02b650e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:16.610057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:16.612090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:16.613265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:16.613265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:16.613265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:16.613265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:16.613265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:16.613265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:16.613265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:16.613265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:16.613265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:55:16.613265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:16.613265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 356/707] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Sep 8 15:55:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:16.620265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe3989-17ef-44eb-9c30-6070b02b650e None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:16.728447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe3989-17ef-44eb-9c30-6070b02b650e None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:16.734181 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:16.746245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b2875-8298-49ca-be73-a9c72889e125 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:16.764679 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50119ef1-025c-45e6-8ec6-6639fcf7a8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:16.765344 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50119ef1-025c-45e6-8ec6-6639fcf7a8db None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:16.765684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50119ef1-025c-45e6-8ec6-6639fcf7a8db None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:16.766162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50119ef1-025c-45e6-8ec6-6639fcf7a8db None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:16.766720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50119ef1-025c-45e6-8ec6-6639fcf7a8db None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:16.767023 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50119ef1-025c-45e6-8ec6-6639fcf7a8db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:16.768551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50119ef1-025c-45e6-8ec6-6639fcf7a8db None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:16.788244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568e6e1-be7f-4105-92e1-27035eb62496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:16.788542 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 357/708] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Sep 8 15:55:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:16.793054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568e6e1-be7f-4105-92e1-27035eb62496 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:16.793054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568e6e1-be7f-4105-92e1-27035eb62496 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:16.793054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568e6e1-be7f-4105-92e1-27035eb62496 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:16.793054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568e6e1-be7f-4105-92e1-27035eb62496 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:16.793054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568e6e1-be7f-4105-92e1-27035eb62496 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:16.793054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568e6e1-be7f-4105-92e1-27035eb62496 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:16.795284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50119ef1-025c-45e6-8ec6-6639fcf7a8db None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:16.809932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 352/709] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:16.815742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568e6e1-be7f-4105-92e1-27035eb62496 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:16.844596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:16.959395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50119ef1-025c-45e6-8ec6-6639fcf7a8db None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:16.978982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568e6e1-be7f-4105-92e1-27035eb62496 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:17.014064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca834349-57fa-489f-a33f-2e834e383a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:17.014681 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca834349-57fa-489f-a33f-2e834e383a05 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:17.014954 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca834349-57fa-489f-a33f-2e834e383a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:17.015219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca834349-57fa-489f-a33f-2e834e383a05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:17.036520 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5bbff-7134-4053-b2c6-9299b69354dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:17.037515 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5bbff-7134-4053-b2c6-9299b69354dc None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:17.038049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5bbff-7134-4053-b2c6-9299b69354dc None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:17.038562 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c5bbff-7134-4053-b2c6-9299b69354dc None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:17.039480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5bbff-7134-4053-b2c6-9299b69354dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:17.041310 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c5bbff-7134-4053-b2c6-9299b69354dc None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:17.069299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 353/710] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:55:16 2022] POST /identity/v3/users => generated 254 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:55:17.070625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5bbff-7134-4053-b2c6-9299b69354dc None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:17.083865 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:17.114482 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca834349-57fa-489f-a33f-2e834e383a05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:17.118766 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca834349-57fa-489f-a33f-2e834e383a05 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:17.185249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:17.200335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c5bbff-7134-4053-b2c6-9299b69354dc None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:17.240674 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e69eb5-28c8-4ba1-839e-01fd92146eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:17.241731 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e69eb5-28c8-4ba1-839e-01fd92146eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:17.242256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e69eb5-28c8-4ba1-839e-01fd92146eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:17.242873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e69eb5-28c8-4ba1-839e-01fd92146eb1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:17.243685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e69eb5-28c8-4ba1-839e-01fd92146eb1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:17.245285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e69eb5-28c8-4ba1-839e-01fd92146eb1 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:17.251163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 358/711] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:55:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-06e69eb5-28c8-4ba1-839e-01fd92146eb1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:17.259708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:17.260918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:17.261814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:17.261814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:17.261814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:17.261814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:17.261814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:17.261814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:17.261814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:17.261814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:17.261814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:17.261814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:17.261814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:17.261814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:17.261814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 15:55:17.261814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:17.264803 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 354/712] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:17 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:17.266909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca834349-57fa-489f-a33f-2e834e383a05 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:17.276833 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e69eb5-28c8-4ba1-839e-01fd92146eb1 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:17.365293 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:17.378493 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca834349-57fa-489f-a33f-2e834e383a05 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:17.400111 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e69eb5-28c8-4ba1-839e-01fd92146eb1 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:17.415604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a998e-199e-49ea-9d02-7ca92a5505fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:17.416256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a998e-199e-49ea-9d02-7ca92a5505fb None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:17.416523 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a998e-199e-49ea-9d02-7ca92a5505fb None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:17.416798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a998e-199e-49ea-9d02-7ca92a5505fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:17.426566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:17.429497 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a998e-199e-49ea-9d02-7ca92a5505fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:17.431490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a998e-199e-49ea-9d02-7ca92a5505fb None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:17.446428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a290a50-af56-438a-846a-38a8f94059d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:17.447191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a290a50-af56-438a-846a-38a8f94059d1 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:17.447607 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a290a50-af56-438a-846a-38a8f94059d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:17.447976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a290a50-af56-438a-846a-38a8f94059d1 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-c01a998e-199e-49ea-9d02-7ca92a5505fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:17.448694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:17.449732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:17.450630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:17.450630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:17.450630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:17.452484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 359/713] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:17.452602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a290a50-af56-438a-846a-38a8f94059d1 None admin] RBAC: Adding query filter params (name=network) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:17.453001 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a290a50-af56-438a-846a-38a8f94059d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:17.454599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a290a50-af56-438a-846a-38a8f94059d1 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:17.460561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a998e-199e-49ea-9d02-7ca92a5505fb None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:17.465845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 355/714] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Sep 8 15:55:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:17.471113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a290a50-af56-438a-846a-38a8f94059d1 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:17.577726 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a998e-199e-49ea-9d02-7ca92a5505fb None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:17.581466 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a290a50-af56-438a-846a-38a8f94059d1 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:17.615393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1cd2f-ace8-498c-8eb6-d315b260bf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:17.616283 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1cd2f-ace8-498c-8eb6-d315b260bf2f None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:17.616700 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1cd2f-ace8-498c-8eb6-d315b260bf2f None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:17.616978 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbc614-736f-4f2a-844d-4ef81d41dfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:17.617295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1cd2f-ace8-498c-8eb6-d315b260bf2f None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:17.617778 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbc614-736f-4f2a-844d-4ef81d41dfaa None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:17.618169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbc614-736f-4f2a-844d-4ef81d41dfaa None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:17.618399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1cd2f-ace8-498c-8eb6-d315b260bf2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:17.618616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdbc614-736f-4f2a-844d-4ef81d41dfaa None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:17.618974 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1cd2f-ace8-498c-8eb6-d315b260bf2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:17.619459 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbc614-736f-4f2a-844d-4ef81d41dfaa None admin] RBAC: Adding query filter params (type=network) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:17.619918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbc614-736f-4f2a-844d-4ef81d41dfaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:17.620816 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1cd2f-ace8-498c-8eb6-d315b260bf2f None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:17.621516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdbc614-736f-4f2a-844d-4ef81d41dfaa None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:17.625105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:17.627993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 356/715] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Sep 8 15:55:17 2022] GET /identity/v3/services?type=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:17.639125 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 360/716] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:17.644496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbc614-736f-4f2a-844d-4ef81d41dfaa None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:17.665180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1583f7-e30c-4a69-a0d9-c7097036e684 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:17.665856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1583f7-e30c-4a69-a0d9-c7097036e684 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:17.666315 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1583f7-e30c-4a69-a0d9-c7097036e684 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:17.667936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 361/717] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:55:17.673926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6568b6a-0c90-4e06-89bf-c737a9c3bac7 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:17.674254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6568b6a-0c90-4e06-89bf-c737a9c3bac7 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:17.674526 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6568b6a-0c90-4e06-89bf-c737a9c3bac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:17.696578 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-b6568b6a-0c90-4e06-89bf-c737a9c3bac7 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:17.698230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b6568b6a-0c90-4e06-89bf-c737a9c3bac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:17.754988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:17.795474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 362/718] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:55:17.795739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdbc614-736f-4f2a-844d-4ef81d41dfaa None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:17.839967 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d37a98-f9ec-4fb1-9c15-76151da6156a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:17.840570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d37a98-f9ec-4fb1-9c15-76151da6156a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:17.840840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d37a98-f9ec-4fb1-9c15-76151da6156a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:17.841158 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d37a98-f9ec-4fb1-9c15-76151da6156a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:17.848748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d37a98-f9ec-4fb1-9c15-76151da6156a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:17.850263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d37a98-f9ec-4fb1-9c15-76151da6156a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:17.863309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-81d37a98-f9ec-4fb1-9c15-76151da6156a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:17.863309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:17.863309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:17.863309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:17.863309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:17.863309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:17.863309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:17.863309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:17.863309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:17.863309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:17.863309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:17.863309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:17.863309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:17.863309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:17.863309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:17.864298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:17.865746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:17.865746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:17.865746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:17.865746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:17.865746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:17.865746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:17.865746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:17.865746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:17.865746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 357/719] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:17.870235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6568b6a-0c90-4e06-89bf-c737a9c3bac7 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:17.973508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6568b6a-0c90-4e06-89bf-c737a9c3bac7 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:18.010384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ca51c-b58a-4e36-87f0-f570647b2463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:18.011331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ca51c-b58a-4e36-87f0-f570647b2463 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:18.011886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ca51c-b58a-4e36-87f0-f570647b2463 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:18.012325 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ca51c-b58a-4e36-87f0-f570647b2463 None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:18.013193 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ca51c-b58a-4e36-87f0-f570647b2463 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:18.013737 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ca51c-b58a-4e36-87f0-f570647b2463 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:18.015332 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54ca51c-b58a-4e36-87f0-f570647b2463 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:18.033794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 363/720] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:18.039795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d37a98-f9ec-4fb1-9c15-76151da6156a None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:18.102317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ca51c-b58a-4e36-87f0-f570647b2463 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:18.120993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:18.147729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:18.188892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d37a98-f9ec-4fb1-9c15-76151da6156a None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:18.206454 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ca51c-b58a-4e36-87f0-f570647b2463 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:18.230219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0906c-43ea-40a1-bd35-cfc740882975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:18.231212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0906c-43ea-40a1-bd35-cfc740882975 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:18.231713 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0906c-43ea-40a1-bd35-cfc740882975 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:18.232183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0906c-43ea-40a1-bd35-cfc740882975 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:18.233233 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0906c-43ea-40a1-bd35-cfc740882975 None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=6e9209e77ce44b128e25a8ddcf78f8b5) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:18.233730 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0906c-43ea-40a1-bd35-cfc740882975 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:18.235530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f0906c-43ea-40a1-bd35-cfc740882975 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:18.240913 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144f385-8da6-4c77-a480-d9c3b5d29821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:18.241526 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144f385-8da6-4c77-a480-d9c3b5d29821 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:18.241810 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144f385-8da6-4c77-a480-d9c3b5d29821 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:18.242070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144f385-8da6-4c77-a480-d9c3b5d29821 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:18.242737 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144f385-8da6-4c77-a480-d9c3b5d29821 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:18.243623 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 358/721] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:55:18 2022] GET /identity/v3/role_assignments?user.id=6e9209e77ce44b128e25a8ddcf78f8b5&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:18.244207 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144f385-8da6-4c77-a480-d9c3b5d29821 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:18.246762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea78122-9c91-4d29-87da-09d11a4b2a69 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:18.247338 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea78122-9c91-4d29-87da-09d11a4b2a69 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:18.247946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea78122-9c91-4d29-87da-09d11a4b2a69 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:18.249950 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 359/722] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:18.255839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379d709-8265-4c5e-96e0-b54eb10b8406 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:18.256427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379d709-8265-4c5e-96e0-b54eb10b8406 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:18.257030 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379d709-8265-4c5e-96e0-b54eb10b8406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:18.269875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 364/723] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Sep 8 15:55:18 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 15:55:18.279385 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-9379d709-8265-4c5e-96e0-b54eb10b8406 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:18.281260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-9379d709-8265-4c5e-96e0-b54eb10b8406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:18.393077 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 360/724] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:18 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:55:18.716115 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144f385-8da6-4c77-a480-d9c3b5d29821 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:18.823640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144f385-8da6-4c77-a480-d9c3b5d29821 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:18.825519 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:18.880529 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20da87-56c1-47e3-8299-18cbb0a57fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:18.881169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20da87-56c1-47e3-8299-18cbb0a57fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:18.881443 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20da87-56c1-47e3-8299-18cbb0a57fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:18.881708 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20da87-56c1-47e3-8299-18cbb0a57fb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:18.893756 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20da87-56c1-47e3-8299-18cbb0a57fb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-9c20da87-56c1-47e3-8299-18cbb0a57fb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:18.899222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:18.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:18.901044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:18.901044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:18.901044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:18.901044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:18.901044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:18.901044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:18.901044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:18.901044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:18.901044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:18.901044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:18.901044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:55:18.901044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:18.908074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379d709-8265-4c5e-96e0-b54eb10b8406 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:18.910170 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 365/725] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:55:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:18.977980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:19.047090 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379d709-8265-4c5e-96e0-b54eb10b8406 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:19.092549 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fce98a-159d-48db-98a3-a4a89618718d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:19.093228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fce98a-159d-48db-98a3-a4a89618718d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:19.093498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fce98a-159d-48db-98a3-a4a89618718d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:19.093759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fce98a-159d-48db-98a3-a4a89618718d None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:19.094491 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fce98a-159d-48db-98a3-a4a89618718d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:19.094790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fce98a-159d-48db-98a3-a4a89618718d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:19.096231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fce98a-159d-48db-98a3-a4a89618718d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:19.111552 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 361/726] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:55:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:19.117195 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20da87-56c1-47e3-8299-18cbb0a57fb2 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:19.240921 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20da87-56c1-47e3-8299-18cbb0a57fb2 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:19.287050 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a757d-7589-4c33-8aef-3fe65cd4ff0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:19.287716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a757d-7589-4c33-8aef-3fe65cd4ff0f None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:19.287988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a757d-7589-4c33-8aef-3fe65cd4ff0f None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:19.288253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a757d-7589-4c33-8aef-3fe65cd4ff0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:19.299222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a757d-7589-4c33-8aef-3fe65cd4ff0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:19.300694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a757d-7589-4c33-8aef-3fe65cd4ff0f None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-290a757d-7589-4c33-8aef-3fe65cd4ff0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:19.321429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:19.322507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:19.323431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:19.323431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:19.323431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:19.325159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 366/727] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:19.330055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fce98a-159d-48db-98a3-a4a89618718d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:19.444789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fce98a-159d-48db-98a3-a4a89618718d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:19.457647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:19.479259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77770b1-448c-4ffd-a650-f13b7cc7e098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:19.479892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77770b1-448c-4ffd-a650-f13b7cc7e098 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:19.480162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77770b1-448c-4ffd-a650-f13b7cc7e098 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:19.480429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77770b1-448c-4ffd-a650-f13b7cc7e098 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:19.481066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77770b1-448c-4ffd-a650-f13b7cc7e098 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:19.481363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77770b1-448c-4ffd-a650-f13b7cc7e098 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:19.482726 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77770b1-448c-4ffd-a650-f13b7cc7e098 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:19.495233 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 362/728] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:19.500688 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a757d-7589-4c33-8aef-3fe65cd4ff0f None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:19.521503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:19.615422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a757d-7589-4c33-8aef-3fe65cd4ff0f None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:19.650041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4fb85-648b-4bb6-a945-2b13b41ae8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:19.650676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4fb85-648b-4bb6-a945-2b13b41ae8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:19.650947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4fb85-648b-4bb6-a945-2b13b41ae8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:19.651211 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4fb85-648b-4bb6-a945-2b13b41ae8f3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:19.658997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4fb85-648b-4bb6-a945-2b13b41ae8f3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:19.660460 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae4fb85-648b-4bb6-a945-2b13b41ae8f3 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-3ae4fb85-648b-4bb6-a945-2b13b41ae8f3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:19.671898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:19.672891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:19.673844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:19.673844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:19.673844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:19.673844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:19.673844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:19.673844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:19.673844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:19.673844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:19.673844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 15:55:19.673844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:19.673844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 367/729] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:55:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:19.684580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77770b1-448c-4ffd-a650-f13b7cc7e098 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:19.805027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77770b1-448c-4ffd-a650-f13b7cc7e098 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:19.853074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46460f7-f98a-4045-958a-bfa1247f8219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:19.853756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46460f7-f98a-4045-958a-bfa1247f8219 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:19.854043 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46460f7-f98a-4045-958a-bfa1247f8219 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:19.854309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46460f7-f98a-4045-958a-bfa1247f8219 None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:19.855049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46460f7-f98a-4045-958a-bfa1247f8219 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:19.855349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46460f7-f98a-4045-958a-bfa1247f8219 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:19.856818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46460f7-f98a-4045-958a-bfa1247f8219 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:19.881301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 363/730] 10.222.0.27 () {56 vars in 1135 bytes} [Thu Sep 8 15:55:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:19.886878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4fb85-648b-4bb6-a945-2b13b41ae8f3 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:19.981152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:20.011488 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4fb85-648b-4bb6-a945-2b13b41ae8f3 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:20.046253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef71ca7c-2ccc-4b3e-9c1e-88c96d63c9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:20.046873 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71ca7c-2ccc-4b3e-9c1e-88c96d63c9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:20.047154 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71ca7c-2ccc-4b3e-9c1e-88c96d63c9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:20.047422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71ca7c-2ccc-4b3e-9c1e-88c96d63c9a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:20.115733 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71ca7c-2ccc-4b3e-9c1e-88c96d63c9a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:20.117132 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef71ca7c-2ccc-4b3e-9c1e-88c96d63c9a1 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:20.204122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:20.218745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 368/731] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:55:19 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 15:55:20.226135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46460f7-f98a-4045-958a-bfa1247f8219 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:20.282148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:20.349359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba5c09-183e-44d4-8b89-370a2390cb09 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:20.349821 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba5c09-183e-44d4-8b89-370a2390cb09 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:20.350099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba5c09-183e-44d4-8b89-370a2390cb09 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:20.351807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 369/732] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:55:20.359012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c6076-4ea1-462a-bef1-6a5823d2c800 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:20.359295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c6076-4ea1-462a-bef1-6a5823d2c800 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:20.359604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78c6076-4ea1-462a-bef1-6a5823d2c800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:20.385171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46460f7-f98a-4045-958a-bfa1247f8219 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:20.391659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-c78c6076-4ea1-462a-bef1-6a5823d2c800 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:20.393312 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c78c6076-4ea1-462a-bef1-6a5823d2c800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:20.428712 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8379f0-c84a-4e17-8bad-5a3d3debd63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:20.429361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8379f0-c84a-4e17-8bad-5a3d3debd63f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:20.429633 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8379f0-c84a-4e17-8bad-5a3d3debd63f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:20.429900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8379f0-c84a-4e17-8bad-5a3d3debd63f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:20.440607 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8379f0-c84a-4e17-8bad-5a3d3debd63f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:20.442133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8379f0-c84a-4e17-8bad-5a3d3debd63f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-7b8379f0-c84a-4e17-8bad-5a3d3debd63f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:20.457255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:20.458480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:20.459656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:20.459656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:20.459656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:20.463626 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 364/733] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:20.464805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4142ac8-1122-43e2-956a-37e752031a6e None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:20.465084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4142ac8-1122-43e2-956a-37e752031a6e None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:20.465352 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4142ac8-1122-43e2-956a-37e752031a6e None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:20.466914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 365/734] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:20.467647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4142ac8-1122-43e2-956a-37e752031a6e None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:20.482866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 370/735] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 15:55:20.488336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371f550-6297-4127-8c5d-e400cc7bd5c9 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:20.488336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371f550-6297-4127-8c5d-e400cc7bd5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:20.488336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371f550-6297-4127-8c5d-e400cc7bd5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:20.519915 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-b371f550-6297-4127-8c5d-e400cc7bd5c9 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:20.521139 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b371f550-6297-4127-8c5d-e400cc7bd5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:20.628761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:20.629457 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 371/736] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 15:55:20.655003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4142ac8-1122-43e2-956a-37e752031a6e None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:20.699319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d31bb-ce89-4981-bd9d-a5e5793143cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:20.699962 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d31bb-ce89-4981-bd9d-a5e5793143cf None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:20.700235 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d31bb-ce89-4981-bd9d-a5e5793143cf None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:20.700551 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d31bb-ce89-4981-bd9d-a5e5793143cf None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:20.701204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d31bb-ce89-4981-bd9d-a5e5793143cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:20.701503 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d31bb-ce89-4981-bd9d-a5e5793143cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:20.702941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d31bb-ce89-4981-bd9d-a5e5793143cf None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:20.722178 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 366/737] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:20.727956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371f550-6297-4127-8c5d-e400cc7bd5c9 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:20.783843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:20.816026 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d31bb-ce89-4981-bd9d-a5e5793143cf None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:20.843486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371f550-6297-4127-8c5d-e400cc7bd5c9 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:20.879389 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e114b42-26f6-4bf0-aea6-d8bbd0aa30e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:20.880120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e114b42-26f6-4bf0-aea6-d8bbd0aa30e0 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:20.880401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e114b42-26f6-4bf0-aea6-d8bbd0aa30e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:20.880687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e114b42-26f6-4bf0-aea6-d8bbd0aa30e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:20.888117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e114b42-26f6-4bf0-aea6-d8bbd0aa30e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:20.889730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e114b42-26f6-4bf0-aea6-d8bbd0aa30e0 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:20.901939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-9e114b42-26f6-4bf0-aea6-d8bbd0aa30e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:20.901939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:20.901939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:20.901939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:20.901939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:20.901939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:20.901939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:20.901939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:20.901939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:20.901939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:20.901939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:20.901939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:20.901939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:20.901939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:20.901939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:20.902894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:20.903878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:20.903878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:20.903878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:20.903878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:20.903878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:20.903878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:20.903878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:20.903878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:20.903878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 372/738] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:20.909239 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afd313-5063-4b92-8dba-388b1784f1b5 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:20.909463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afd313-5063-4b92-8dba-388b1784f1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:20.909739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afd313-5063-4b92-8dba-388b1784f1b5 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:20.911304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 373/739] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:20.912776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afd313-5063-4b92-8dba-388b1784f1b5 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:20.971218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d31bb-ce89-4981-bd9d-a5e5793143cf None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:21.008377 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416923ed-b87a-4eac-aaa0-ef4f9dc65ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:21.008985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416923ed-b87a-4eac-aaa0-ef4f9dc65ef5 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:21.009279 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416923ed-b87a-4eac-aaa0-ef4f9dc65ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:21.009512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416923ed-b87a-4eac-aaa0-ef4f9dc65ef5 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:21.010177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416923ed-b87a-4eac-aaa0-ef4f9dc65ef5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:21.011573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416923ed-b87a-4eac-aaa0-ef4f9dc65ef5 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:21.012809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:21.022463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afd313-5063-4b92-8dba-388b1784f1b5 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:21.028900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 367/740] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Sep 8 15:55:20 2022] POST /identity/v3/services => generated 240 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 15:55:21.037093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341f55a-cfbe-4c51-91d5-df71693fcba9 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:21.037774 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341f55a-cfbe-4c51-91d5-df71693fcba9 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:21.038278 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341f55a-cfbe-4c51-91d5-df71693fcba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:21.045593 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:21.061177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-3341f55a-cfbe-4c51-91d5-df71693fcba9 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:21.063023 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3341f55a-cfbe-4c51-91d5-df71693fcba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:21.087092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebc11d-d6d1-4aec-9ee9-d513de3a2345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:21.087813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebc11d-d6d1-4aec-9ee9-d513de3a2345 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:21.088086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebc11d-d6d1-4aec-9ee9-d513de3a2345 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:21.088356 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bebc11d-d6d1-4aec-9ee9-d513de3a2345 None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:21.089299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bebc11d-d6d1-4aec-9ee9-d513de3a2345 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:21.089611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bebc11d-d6d1-4aec-9ee9-d513de3a2345 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:21.091429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bebc11d-d6d1-4aec-9ee9-d513de3a2345 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:21.111255 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 374/741] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:21.111255 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebc11d-d6d1-4aec-9ee9-d513de3a2345 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:21.151626 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 368/742] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:21.152931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341f55a-cfbe-4c51-91d5-df71693fcba9 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:21.268228 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bebc11d-d6d1-4aec-9ee9-d513de3a2345 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:21.274057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341f55a-cfbe-4c51-91d5-df71693fcba9 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:21.308977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41843595-5057-448b-8763-afbacb00c98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:21.309646 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41843595-5057-448b-8763-afbacb00c98f None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:21.309927 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41843595-5057-448b-8763-afbacb00c98f None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:21.310201 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41843595-5057-448b-8763-afbacb00c98f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:21.312698 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:21.318258 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b0391-d9d9-4ffd-95f3-b50a8c427932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:21.319412 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b0391-d9d9-4ffd-95f3-b50a8c427932 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:21.320039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b0391-d9d9-4ffd-95f3-b50a8c427932 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:21.320517 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b0391-d9d9-4ffd-95f3-b50a8c427932 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:21.321592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b0391-d9d9-4ffd-95f3-b50a8c427932 None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=de54687547a14ab89cc41a1fabaab400) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:21.322111 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b0391-d9d9-4ffd-95f3-b50a8c427932 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:21.323851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b0391-d9d9-4ffd-95f3-b50a8c427932 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:21.324400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41843595-5057-448b-8763-afbacb00c98f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-41843595-5057-448b-8763-afbacb00c98f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:21.330937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:21.331973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:21.332830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:21.332830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:21.332830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:21.332830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:21.332830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:21.332830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:21.332830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:21.332830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:21.332830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:21.332830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:21.332830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:21.332830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:21.333470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 375/743] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:55:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:21.334298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 369/744] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:55:21 2022] GET /identity/v3/role_assignments?user.id=de54687547a14ab89cc41a1fabaab400&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:21.339498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41843595-5057-448b-8763-afbacb00c98f None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:21.354734 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:21.429762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956fcb4-1ac1-4b31-abca-1ebec408c40d None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:21.430096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956fcb4-1ac1-4b31-abca-1ebec408c40d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:21.430298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956fcb4-1ac1-4b31-abca-1ebec408c40d None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:21.431986 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 370/745] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:21.438432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0f266-9fdc-45a1-9e73-2b2e7da26359 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:21.438708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0f266-9fdc-45a1-9e73-2b2e7da26359 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:21.438908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f0f266-9fdc-45a1-9e73-2b2e7da26359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:21.457159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41843595-5057-448b-8763-afbacb00c98f None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:21.460052 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-99f0f266-9fdc-45a1-9e73-2b2e7da26359 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:21.462463 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-99f0f266-9fdc-45a1-9e73-2b2e7da26359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:21.494692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe71a-7534-4150-85aa-278a109908b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:21.495436 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe71a-7534-4150-85aa-278a109908b1 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:21.495740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe71a-7534-4150-85aa-278a109908b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:21.496011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bbe71a-7534-4150-85aa-278a109908b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:21.496709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbe71a-7534-4150-85aa-278a109908b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:21.497116 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbe71a-7534-4150-85aa-278a109908b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:21.499162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bbe71a-7534-4150-85aa-278a109908b1 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:21.509724 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 376/746] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:55:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:21.510923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe71a-7534-4150-85aa-278a109908b1 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:21.540266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 371/747] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:21.548619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0f266-9fdc-45a1-9e73-2b2e7da26359 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:21.617043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:21.651565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f0f266-9fdc-45a1-9e73-2b2e7da26359 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:21.652570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bbe71a-7534-4150-85aa-278a109908b1 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:21.679508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:21.692226 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f2560-95fc-4b3f-b557-fb2ce5dcfd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:21.694121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f2560-95fc-4b3f-b557-fb2ce5dcfd87 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:21.694771 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f2560-95fc-4b3f-b557-fb2ce5dcfd87 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:21.696341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f2560-95fc-4b3f-b557-fb2ce5dcfd87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:21.716797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f2560-95fc-4b3f-b557-fb2ce5dcfd87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:21.719133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f2560-95fc-4b3f-b557-fb2ce5dcfd87 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:21.719994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b20d9-9d4b-49f5-98ef-eb249f2c484e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:21.720971 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b20d9-9d4b-49f5-98ef-eb249f2c484e None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:21.721445 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b20d9-9d4b-49f5-98ef-eb249f2c484e None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:21.722361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b20d9-9d4b-49f5-98ef-eb249f2c484e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:21.741597 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b20d9-9d4b-49f5-98ef-eb249f2c484e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-2b6f2560-95fc-4b3f-b557-fb2ce5dcfd87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:21.746931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:21.747916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:21.748826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:21.748826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:21.748826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-914b20d9-9d4b-49f5-98ef-eb249f2c484e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:21.755006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:21.755988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:21.756937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:21.756937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:21.756937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:21.756937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:21.756937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:21.756937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:21.756937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:21.756937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:21.756937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:21.756937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:21.756937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:55:21.756937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:21.759727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 372/748] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:21.761552 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 377/749] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:55:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:21.769504 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f2560-95fc-4b3f-b557-fb2ce5dcfd87 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:21.769866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b20d9-9d4b-49f5-98ef-eb249f2c484e None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:21.933003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f2560-95fc-4b3f-b557-fb2ce5dcfd87 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:21.933899 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b20d9-9d4b-49f5-98ef-eb249f2c484e None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:21.975908 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:21.977633 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5f6bd-6687-49c4-be3c-b31e78a4dc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:21.978525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5f6bd-6687-49c4-be3c-b31e78a4dc89 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:21.978993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5f6bd-6687-49c4-be3c-b31e78a4dc89 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:21.979580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5f6bd-6687-49c4-be3c-b31e78a4dc89 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:21.980527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5f6bd-6687-49c4-be3c-b31e78a4dc89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:21.981010 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5f6bd-6687-49c4-be3c-b31e78a4dc89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:21.981117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65762eda-3c03-4a39-b3b6-2742c2a2f531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:21.981823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65762eda-3c03-4a39-b3b6-2742c2a2f531 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:21.982166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65762eda-3c03-4a39-b3b6-2742c2a2f531 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:21.982510 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65762eda-3c03-4a39-b3b6-2742c2a2f531 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:21.983248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65762eda-3c03-4a39-b3b6-2742c2a2f531 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:21.983789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5f6bd-6687-49c4-be3c-b31e78a4dc89 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:21.984055 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65762eda-3c03-4a39-b3b6-2742c2a2f531 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:21.985902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65762eda-3c03-4a39-b3b6-2742c2a2f531 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:21.997441 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 378/750] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:55:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:21.998414 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65762eda-3c03-4a39-b3b6-2742c2a2f531 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:21.999225 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 373/751] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:22.003172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5f6bd-6687-49c4-be3c-b31e78a4dc89 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:22.047031 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:22.120687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5f6bd-6687-49c4-be3c-b31e78a4dc89 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:22.138520 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65762eda-3c03-4a39-b3b6-2742c2a2f531 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:22.156301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd8363-78ae-487a-9f14-23f6241bdf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:22.157221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd8363-78ae-487a-9f14-23f6241bdf2e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:22.157659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd8363-78ae-487a-9f14-23f6241bdf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:22.158071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd8363-78ae-487a-9f14-23f6241bdf2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:22.172940 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd8363-78ae-487a-9f14-23f6241bdf2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:22.174521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd8363-78ae-487a-9f14-23f6241bdf2e None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:22.186956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434862a-b200-4de7-9e87-36043035559c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:22.187730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434862a-b200-4de7-9e87-36043035559c None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:22.188083 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434862a-b200-4de7-9e87-36043035559c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:22.188424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434862a-b200-4de7-9e87-36043035559c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:22.189109 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434862a-b200-4de7-9e87-36043035559c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:22.190697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434862a-b200-4de7-9e87-36043035559c None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-34fd8363-78ae-487a-9f14-23f6241bdf2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:22.203231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:22.204335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:22.205370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:22.205370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:22.205370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-b434862a-b200-4de7-9e87-36043035559c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:22.205556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:22.206463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:22.207324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:22.207324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:22.207324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:22.207324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:22.207324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:22.207324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:22.207324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:22.207324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:22.207324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:22.207324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:22.207324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:22.207324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:22.207324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 15:55:22.207324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:22.210410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 379/752] 10.222.0.27 () {56 vars in 1129 bytes} [Thu Sep 8 15:55:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:22.210794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 374/753] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:22.215320 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434862a-b200-4de7-9e87-36043035559c None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:22.215532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd8363-78ae-487a-9f14-23f6241bdf2e None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:22.284216 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:22.327748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:22.340380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434862a-b200-4de7-9e87-36043035559c None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:22.343925 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd8363-78ae-487a-9f14-23f6241bdf2e None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:22.378382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837c9e8-7caf-4b61-8216-f40e562c683c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:22.379027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837c9e8-7caf-4b61-8216-f40e562c683c None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:22.379306 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837c9e8-7caf-4b61-8216-f40e562c683c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:22.379615 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837c9e8-7caf-4b61-8216-f40e562c683c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:22.384686 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76080043-45a8-4a4b-bce2-5d5af990c8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:22.385529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76080043-45a8-4a4b-bce2-5d5af990c8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:22.385993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76080043-45a8-4a4b-bce2-5d5af990c8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:22.386452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76080043-45a8-4a4b-bce2-5d5af990c8a7 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:22.386972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837c9e8-7caf-4b61-8216-f40e562c683c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:22.387513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76080043-45a8-4a4b-bce2-5d5af990c8a7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:22.388019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76080043-45a8-4a4b-bce2-5d5af990c8a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:22.388515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837c9e8-7caf-4b61-8216-f40e562c683c None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:22.389788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76080043-45a8-4a4b-bce2-5d5af990c8a7 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:22.394926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 375/754] 10.222.0.27 () {56 vars in 1124 bytes} [Thu Sep 8 15:55:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:22.395953 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76080043-45a8-4a4b-bce2-5d5af990c8a7 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-1837c9e8-7caf-4b61-8216-f40e562c683c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:22.405891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:22.406846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:22.407836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:22.407836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:22.407836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:22.407836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:22.407836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:22.407836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:22.407836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:22.407836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:22.407836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 15:55:22.407836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:22.412069 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 380/755] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Sep 8 15:55:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:22.412895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837c9e8-7caf-4b61-8216-f40e562c683c None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:22.498941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76080043-45a8-4a4b-bce2-5d5af990c8a7 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:22.523601 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837c9e8-7caf-4b61-8216-f40e562c683c None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:22.533456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2c036-563c-4a28-ba27-fd871aeb1db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:22.534417 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2c036-563c-4a28-ba27-fd871aeb1db2 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:22.534918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2c036-563c-4a28-ba27-fd871aeb1db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:22.535386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2c036-563c-4a28-ba27-fd871aeb1db2 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:22.536258 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2c036-563c-4a28-ba27-fd871aeb1db2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:22.536746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2c036-563c-4a28-ba27-fd871aeb1db2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:22.538298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2c036-563c-4a28-ba27-fd871aeb1db2 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:22.542251 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:22.556028 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 376/756] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:22.556687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2c036-563c-4a28-ba27-fd871aeb1db2 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:22.563327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2468f6-9564-4238-af28-807fb1769b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:22.564037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2468f6-9564-4238-af28-807fb1769b6a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:22.564319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2468f6-9564-4238-af28-807fb1769b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:22.564626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2468f6-9564-4238-af28-807fb1769b6a None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:22.565322 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2468f6-9564-4238-af28-807fb1769b6a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:22.565626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2468f6-9564-4238-af28-807fb1769b6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:22.567080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2468f6-9564-4238-af28-807fb1769b6a None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:22.578632 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 381/757] 10.222.0.27 () {56 vars in 1124 bytes} [Thu Sep 8 15:55:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:22.579829 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:22.579947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2468f6-9564-4238-af28-807fb1769b6a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:22.668915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2c036-563c-4a28-ba27-fd871aeb1db2 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:22.703714 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c713e-af60-410a-a224-685336b7254d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:22.704329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c713e-af60-410a-a224-685336b7254d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:22.704603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c713e-af60-410a-a224-685336b7254d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:22.704869 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c713e-af60-410a-a224-685336b7254d None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:22.705656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c713e-af60-410a-a224-685336b7254d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:22.705956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c713e-af60-410a-a224-685336b7254d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:22.707695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c713e-af60-410a-a224-685336b7254d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:22.723139 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2468f6-9564-4238-af28-807fb1769b6a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:22.725754 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 377/758] 10.222.0.27 () {56 vars in 1149 bytes} [Thu Sep 8 15:55:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:22.726673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c713e-af60-410a-a224-685336b7254d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:22.764115 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc526f-23a1-4ecc-a10e-ae1482fa5214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:22.765418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc526f-23a1-4ecc-a10e-ae1482fa5214 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:22.765418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc526f-23a1-4ecc-a10e-ae1482fa5214 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:22.765418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc526f-23a1-4ecc-a10e-ae1482fa5214 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:22.773588 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc526f-23a1-4ecc-a10e-ae1482fa5214 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:22.774897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddc526f-23a1-4ecc-a10e-ae1482fa5214 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-eddc526f-23a1-4ecc-a10e-ae1482fa5214 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:22.786338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:22.787479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:22.788647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:22.788647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:22.788647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:22.788647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:22.788647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:22.788647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:22.788647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:22.788647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:22.788647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:55:22.788647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:22.794717 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 382/759] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Sep 8 15:55:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:22.795757 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc526f-23a1-4ecc-a10e-ae1482fa5214 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:22.809663 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:22.843495 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c713e-af60-410a-a224-685336b7254d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:22.848636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:22.880996 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56cd5f-549d-470a-b16c-8405d4c6525e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:22.881841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56cd5f-549d-470a-b16c-8405d4c6525e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:22.882283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56cd5f-549d-470a-b16c-8405d4c6525e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:22.882736 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56cd5f-549d-470a-b16c-8405d4c6525e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:22.883650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56cd5f-549d-470a-b16c-8405d4c6525e None admin] RBAC: Adding query filter params (interface=public, service_id=decaf8abee1545b882a308e70b01b7d7, region_id=RegionOne) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:22.884176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56cd5f-549d-470a-b16c-8405d4c6525e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:22.885733 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b56cd5f-549d-470a-b16c-8405d4c6525e None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:22.894004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 378/760] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Sep 8 15:55:22 2022] GET /identity/v3/endpoints?service_id=decaf8abee1545b882a308e70b01b7d7&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:22.895028 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56cd5f-549d-470a-b16c-8405d4c6525e None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:22.916159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc526f-23a1-4ecc-a10e-ae1482fa5214 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:22.953495 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a7213-4ed6-42aa-b29a-506a65b940e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:22.954166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a7213-4ed6-42aa-b29a-506a65b940e2 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:22.954390 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a7213-4ed6-42aa-b29a-506a65b940e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:22.954645 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a7213-4ed6-42aa-b29a-506a65b940e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:22.999846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56cd5f-549d-470a-b16c-8405d4c6525e None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:23.033667 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a7213-4ed6-42aa-b29a-506a65b940e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:23.035150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a7213-4ed6-42aa-b29a-506a65b940e2 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:23.035770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee09b6e-72ac-457d-b852-7857ec9e4ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:23.036656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee09b6e-72ac-457d-b852-7857ec9e4ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:23.037182 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee09b6e-72ac-457d-b852-7857ec9e4ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:23.037652 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee09b6e-72ac-457d-b852-7857ec9e4ef3 None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:23.038620 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee09b6e-72ac-457d-b852-7857ec9e4ef3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:23.039129 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee09b6e-72ac-457d-b852-7857ec9e4ef3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:23.040801 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee09b6e-72ac-457d-b852-7857ec9e4ef3 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:23.043827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:23.062707 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 379/761] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Sep 8 15:55:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:23.063999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee09b6e-72ac-457d-b852-7857ec9e4ef3 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:23.082559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:23.159096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 383/762] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:55:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:55:23.160331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a7213-4ed6-42aa-b29a-506a65b940e2 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:23.177383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee09b6e-72ac-457d-b852-7857ec9e4ef3 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:23.217917 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10572f0f-2674-4cb7-99e1-a9a19d786264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:23.218532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10572f0f-2674-4cb7-99e1-a9a19d786264 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:23.219114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10572f0f-2674-4cb7-99e1-a9a19d786264 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:23.219403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10572f0f-2674-4cb7-99e1-a9a19d786264 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:23.220038 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10572f0f-2674-4cb7-99e1-a9a19d786264 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:23.222281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10572f0f-2674-4cb7-99e1-a9a19d786264 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:23.229338 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 380/763] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Sep 8 15:55:23 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:23.230389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10572f0f-2674-4cb7-99e1-a9a19d786264 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:23.280053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a7213-4ed6-42aa-b29a-506a65b940e2 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:23.298083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:23.319639 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6e9f1-5741-4780-a6fe-206208b9f71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:23.320696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6e9f1-5741-4780-a6fe-206208b9f71d None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:23.321173 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6e9f1-5741-4780-a6fe-206208b9f71d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:23.321630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6e9f1-5741-4780-a6fe-206208b9f71d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:23.333112 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6e9f1-5741-4780-a6fe-206208b9f71d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:23.334936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6e9f1-5741-4780-a6fe-206208b9f71d None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:23.337994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-9bb6e9f1-5741-4780-a6fe-206208b9f71d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:23.357119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:23.358169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:23.359086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:23.359086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:23.359086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:23.359305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10572f0f-2674-4cb7-99e1-a9a19d786264 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:23.362234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 384/764] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:23.367970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6e9f1-5741-4780-a6fe-206208b9f71d None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:23.441594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1752c1d4-25fb-4010-90ac-97743dc8ff03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:23.445314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752c1d4-25fb-4010-90ac-97743dc8ff03 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:23.445942 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752c1d4-25fb-4010-90ac-97743dc8ff03 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:23.446430 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1752c1d4-25fb-4010-90ac-97743dc8ff03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:23.470398 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752c1d4-25fb-4010-90ac-97743dc8ff03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:23.471987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1752c1d4-25fb-4010-90ac-97743dc8ff03 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-1752c1d4-25fb-4010-90ac-97743dc8ff03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:23.488995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:23.490011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:23.490907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:23.490907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:23.490907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:23.490907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 381/765] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:23.491707 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5fd74-2f85-49c9-96ec-a542019991d4 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:23.492960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5fd74-2f85-49c9-96ec-a542019991d4 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:23.492960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5fd74-2f85-49c9-96ec-a542019991d4 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:23.494081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 382/766] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:23.498409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5fd74-2f85-49c9-96ec-a542019991d4 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:23.539132 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6e9f1-5741-4780-a6fe-206208b9f71d None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:23.579307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948aa39-4274-4270-afcc-f7a4586db0de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:23.579866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948aa39-4274-4270-afcc-f7a4586db0de None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:23.580196 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948aa39-4274-4270-afcc-f7a4586db0de None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:23.580468 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948aa39-4274-4270-afcc-f7a4586db0de None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:23.581109 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948aa39-4274-4270-afcc-f7a4586db0de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:23.581404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948aa39-4274-4270-afcc-f7a4586db0de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:23.582771 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948aa39-4274-4270-afcc-f7a4586db0de None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:23.595866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 385/767] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:23.597217 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd35ea2-e0d0-4656-8d65-2d1b215eb0cc None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:23.597496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd35ea2-e0d0-4656-8d65-2d1b215eb0cc None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:23.597762 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd35ea2-e0d0-4656-8d65-2d1b215eb0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:23.617467 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-8fd35ea2-e0d0-4656-8d65-2d1b215eb0cc None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:23.618706 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-8fd35ea2-e0d0-4656-8d65-2d1b215eb0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:23.637755 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5fd74-2f85-49c9-96ec-a542019991d4 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:23.649657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:23.677842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90c545-2668-4de0-93bc-dcc6831e974f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:23.678498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90c545-2668-4de0-93bc-dcc6831e974f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:23.678907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90c545-2668-4de0-93bc-dcc6831e974f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:23.679268 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb90c545-2668-4de0-93bc-dcc6831e974f None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:23.680097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90c545-2668-4de0-93bc-dcc6831e974f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:23.680490 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90c545-2668-4de0-93bc-dcc6831e974f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:23.681138 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:23.683133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb90c545-2668-4de0-93bc-dcc6831e974f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:23.698206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 383/768] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:23.699312 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90c545-2668-4de0-93bc-dcc6831e974f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:23.707168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 386/769] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:23.708409 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd35ea2-e0d0-4656-8d65-2d1b215eb0cc None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:23.815556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd35ea2-e0d0-4656-8d65-2d1b215eb0cc None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:23.822887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb90c545-2668-4de0-93bc-dcc6831e974f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:23.850792 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90acf8b0-dff1-418d-bfd9-bccc06e61636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:23.851554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acf8b0-dff1-418d-bfd9-bccc06e61636 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:23.851923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acf8b0-dff1-418d-bfd9-bccc06e61636 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:23.852285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90acf8b0-dff1-418d-bfd9-bccc06e61636 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:23.859367 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90acf8b0-dff1-418d-bfd9-bccc06e61636 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:23.860947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90acf8b0-dff1-418d-bfd9-bccc06e61636 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:23.865970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49291a9f-0d62-4f85-a41b-4620bde62b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:23.866729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49291a9f-0d62-4f85-a41b-4620bde62b23 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:23.867064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49291a9f-0d62-4f85-a41b-4620bde62b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:23.867775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49291a9f-0d62-4f85-a41b-4620bde62b23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:23.873270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-90acf8b0-dff1-418d-bfd9-bccc06e61636 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:23.873270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:23.873270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:23.873270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:23.873270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:23.873270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:23.873270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:23.873270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:23.873270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:23.873270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:23.873270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:23.873270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:23.873270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:23.873270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:23.873270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:23.874243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:23.875172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:23.875172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:23.875172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:23.875172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:23.875172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:23.875172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:23.875172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:23.875172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:23.875172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 387/770] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:23.879466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138b6b6-8bb2-4996-9f08-c80724cdecab None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:23.879466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138b6b6-8bb2-4996-9f08-c80724cdecab None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:23.879466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138b6b6-8bb2-4996-9f08-c80724cdecab None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:23.879466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 388/771] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:23.880432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138b6b6-8bb2-4996-9f08-c80724cdecab None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:23.884986 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49291a9f-0d62-4f85-a41b-4620bde62b23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:23.886505 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49291a9f-0d62-4f85-a41b-4620bde62b23 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:23.903060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-49291a9f-0d62-4f85-a41b-4620bde62b23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:23.903060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:23.903060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:23.903060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:23.903060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:23.903060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:23.903060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:23.903060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:23.903060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:23.903060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:23.903060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:23.903060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:23.903060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:23.903060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:23.903060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:23.904363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:23.905788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:23.905788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:23.905788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:23.905788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:23.905788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:23.905788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:23.905788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:23.905788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:23.905788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 384/772] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:23.906599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567518cf-d034-4b8e-9224-8213d87e9ed7 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:23.911467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567518cf-d034-4b8e-9224-8213d87e9ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:23.911467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567518cf-d034-4b8e-9224-8213d87e9ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:23.931900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:23.933263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-567518cf-d034-4b8e-9224-8213d87e9ed7 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:23.935088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-567518cf-d034-4b8e-9224-8213d87e9ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:23.966528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:23.993122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138b6b6-8bb2-4996-9f08-c80724cdecab None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:24.014235 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 385/773] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:24.015427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567518cf-d034-4b8e-9224-8213d87e9ed7 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:24.032474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb0b7a-4b8b-43f7-80f6-c7402cf72937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:24.033563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb0b7a-4b8b-43f7-80f6-c7402cf72937 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:24.034077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb0b7a-4b8b-43f7-80f6-c7402cf72937 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:24.034548 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb0b7a-4b8b-43f7-80f6-c7402cf72937 None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:24.035430 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb0b7a-4b8b-43f7-80f6-c7402cf72937 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:24.036060 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb0b7a-4b8b-43f7-80f6-c7402cf72937 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:24.037778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb0b7a-4b8b-43f7-80f6-c7402cf72937 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:24.051799 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 389/774] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:24.052959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb0b7a-4b8b-43f7-80f6-c7402cf72937 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:24.120881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567518cf-d034-4b8e-9224-8213d87e9ed7 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:24.156028 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb39113-d758-4da3-a953-d5eaa25028fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:24.157051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb39113-d758-4da3-a953-d5eaa25028fd None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:24.157518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb39113-d758-4da3-a953-d5eaa25028fd None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:24.157976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb39113-d758-4da3-a953-d5eaa25028fd None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:24.158886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb39113-d758-4da3-a953-d5eaa25028fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:24.159376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb39113-d758-4da3-a953-d5eaa25028fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:24.161095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb39113-d758-4da3-a953-d5eaa25028fd None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:24.174684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb0b7a-4b8b-43f7-80f6-c7402cf72937 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:24.176635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 386/775] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:24.177685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb39113-d758-4da3-a953-d5eaa25028fd None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:24.182868 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:24.204463 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:24.294829 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb39113-d758-4da3-a953-d5eaa25028fd None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:24.332806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b984c0-8db3-437f-90f8-ca241bc2b1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:24.333419 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b984c0-8db3-437f-90f8-ca241bc2b1fe None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:24.333690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b984c0-8db3-437f-90f8-ca241bc2b1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:24.333955 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b984c0-8db3-437f-90f8-ca241bc2b1fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:24.334677 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b984c0-8db3-437f-90f8-ca241bc2b1fe None admin] RBAC: Adding query filter params (role.id=b256252b45e541a491376a134f502183, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=b4324e4c04e343c98d9bb216c542fcba) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:24.334979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b984c0-8db3-437f-90f8-ca241bc2b1fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:24.336445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b984c0-8db3-437f-90f8-ca241bc2b1fe None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:24.343599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 387/776] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:55:24 2022] GET /identity/v3/role_assignments?user.id=b4324e4c04e343c98d9bb216c542fcba&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=b256252b45e541a491376a134f502183 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:24.344919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b984c0-8db3-437f-90f8-ca241bc2b1fe None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:24.358638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf15b4-6085-4882-bf8e-6fa46493b35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:24.359319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf15b4-6085-4882-bf8e-6fa46493b35f None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:24.359623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf15b4-6085-4882-bf8e-6fa46493b35f None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:24.359893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bf15b4-6085-4882-bf8e-6fa46493b35f None admin] PATH_INFO: `/v3/services/network` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:24.361489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf15b4-6085-4882-bf8e-6fa46493b35f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:24.362012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bf15b4-6085-4882-bf8e-6fa46493b35f None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-f3bf15b4-6085-4882-bf8e-6fa46493b35f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:24.372104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:24.373166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:24.374057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:24.374057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:24.374057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:24.374057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:24.374057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:24.374057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:24.374057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:24.374057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:24.374057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:24.374057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:24.374057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:24.374057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:24.374057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 15:55:24.374057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:24.374057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 390/777] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:24 2022] GET /identity/v3/services/network => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:24.374869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf15b4-6085-4882-bf8e-6fa46493b35f None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:24.487501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:24.495771 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bf15b4-6085-4882-bf8e-6fa46493b35f None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:24.506003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b984c0-8db3-437f-90f8-ca241bc2b1fe None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:24.542636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64aa27b-dd2c-4fb0-8dd0-bcc2e5426772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:24.543606 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64aa27b-dd2c-4fb0-8dd0-bcc2e5426772 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:24.543879 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64aa27b-dd2c-4fb0-8dd0-bcc2e5426772 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:24.544146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64aa27b-dd2c-4fb0-8dd0-bcc2e5426772 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:24.555099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e0139-4aae-40f5-86e9-82e842a70192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:24.556130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64aa27b-dd2c-4fb0-8dd0-bcc2e5426772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:24.556503 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e0139-4aae-40f5-86e9-82e842a70192 None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:24.556979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e0139-4aae-40f5-86e9-82e842a70192 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:24.557486 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07e0139-4aae-40f5-86e9-82e842a70192 None admin] PATH_INFO: `/v3/projects/a825a0b1238d4382995da440944f453b/users/6e9209e77ce44b128e25a8ddcf78f8b5/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:24.559688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e0139-4aae-40f5-86e9-82e842a70192 None admin] RBAC: Authorizing `identity:create_grant(project_id=a825a0b1238d4382995da440944f453b, user_id=6e9209e77ce44b128e25a8ddcf78f8b5, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-d64aa27b-dd2c-4fb0-8dd0-bcc2e5426772 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:24.561174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:24.562150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:24.563010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:24.563010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:24.563010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:24.563010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:24.563010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:24.563010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:24.563010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:24.563010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:24.563010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:24.563010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:24.563010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:55:24.563010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:24.563010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 391/778] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:55:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:24.563730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64aa27b-dd2c-4fb0-8dd0-bcc2e5426772 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:24.569231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07e0139-4aae-40f5-86e9-82e842a70192 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:24.593123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 388/779] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:55:24 2022] PUT /identity/v3/projects/a825a0b1238d4382995da440944f453b/users/6e9209e77ce44b128e25a8ddcf78f8b5/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:55:24.593977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e0139-4aae-40f5-86e9-82e842a70192 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:24.602255 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:24.746522 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64aa27b-dd2c-4fb0-8dd0-bcc2e5426772 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:24.778291 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07e0139-4aae-40f5-86e9-82e842a70192 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:24.819012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b693cde-8013-4540-823e-948799809916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:24.819760 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b693cde-8013-4540-823e-948799809916 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:24.820036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b693cde-8013-4540-823e-948799809916 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:24.820303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b693cde-8013-4540-823e-948799809916 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:24.821055 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b693cde-8013-4540-823e-948799809916 None admin] RBAC: Adding query filter params (name=network) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:24.821361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b693cde-8013-4540-823e-948799809916 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:24.822743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b693cde-8013-4540-823e-948799809916 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:24.827372 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24916ad7-d5be-4f11-a64e-9003134e47c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:24.827950 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 392/780] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Sep 8 15:55:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:24.828503 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24916ad7-d5be-4f11-a64e-9003134e47c9 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:24.828904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24916ad7-d5be-4f11-a64e-9003134e47c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:24.829176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24916ad7-d5be-4f11-a64e-9003134e47c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:24.831286 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24916ad7-d5be-4f11-a64e-9003134e47c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:24.831286 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24916ad7-d5be-4f11-a64e-9003134e47c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:24.831652 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24916ad7-d5be-4f11-a64e-9003134e47c9 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:24.834016 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b693cde-8013-4540-823e-948799809916 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:24.841732 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 389/781] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:55:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:24.847756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24916ad7-d5be-4f11-a64e-9003134e47c9 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:24.858483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:24.943598 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:24.955024 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b693cde-8013-4540-823e-948799809916 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:24.957210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24916ad7-d5be-4f11-a64e-9003134e47c9 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:25.003682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f92b7-6611-4a4c-aa17-254eb5663716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:25.004927 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f92b7-6611-4a4c-aa17-254eb5663716 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:25.005428 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f92b7-6611-4a4c-aa17-254eb5663716 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:25.005876 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f92b7-6611-4a4c-aa17-254eb5663716 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:25.006706 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f92b7-6611-4a4c-aa17-254eb5663716 None admin] RBAC: Adding query filter params (type=network) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:25.008181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f92b7-6611-4a4c-aa17-254eb5663716 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:25.009741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f92b7-6611-4a4c-aa17-254eb5663716 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:25.019583 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 393/782] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Sep 8 15:55:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:25.025826 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f92b7-6611-4a4c-aa17-254eb5663716 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:25.032161 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a18746-607a-4861-9041-362fa67200fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:25.033065 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a18746-607a-4861-9041-362fa67200fd None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:25.033584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a18746-607a-4861-9041-362fa67200fd None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:25.034051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a18746-607a-4861-9041-362fa67200fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:25.056426 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a18746-607a-4861-9041-362fa67200fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:25.059120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a18746-607a-4861-9041-362fa67200fd None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-c0a18746-607a-4861-9041-362fa67200fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:25.082859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:25.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:25.084830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:25.084830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:25.084830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:25.095059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 390/783] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:25.105794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a18746-607a-4861-9041-362fa67200fd None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:25.198699 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:25.223544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f92b7-6611-4a4c-aa17-254eb5663716 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:25.276903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a18746-607a-4861-9041-362fa67200fd None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:25.295311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebdc2b-78f6-4799-949e-5ba02c1daf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:25.295965 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebdc2b-78f6-4799-949e-5ba02c1daf6c None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:25.296242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebdc2b-78f6-4799-949e-5ba02c1daf6c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:25.296508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cebdc2b-78f6-4799-949e-5ba02c1daf6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:25.297218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cebdc2b-78f6-4799-949e-5ba02c1daf6c None admin] RBAC: Adding query filter params (interface=public, service_id=79b90585dc8945adae6e11b007d51f41, region_id=RegionOne) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:25.297522 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cebdc2b-78f6-4799-949e-5ba02c1daf6c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:25.298926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cebdc2b-78f6-4799-949e-5ba02c1daf6c None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:25.322627 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 394/784] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Sep 8 15:55:25 2022] GET /identity/v3/endpoints?service_id=79b90585dc8945adae6e11b007d51f41&interface=public®ion_id=RegionOne => generated 194 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:25.329451 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebdc2b-78f6-4799-949e-5ba02c1daf6c None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:25.351060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dc126-8961-4caa-a4c9-0693b15cca26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:25.351756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dc126-8961-4caa-a4c9-0693b15cca26 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:25.352093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dc126-8961-4caa-a4c9-0693b15cca26 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:25.352370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4dc126-8961-4caa-a4c9-0693b15cca26 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:25.353027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4dc126-8961-4caa-a4c9-0693b15cca26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:25.353340 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4dc126-8961-4caa-a4c9-0693b15cca26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:25.354726 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4dc126-8961-4caa-a4c9-0693b15cca26 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:25.368466 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 391/785] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:25.374856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dc126-8961-4caa-a4c9-0693b15cca26 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:25.411067 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:25.538467 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cebdc2b-78f6-4799-949e-5ba02c1daf6c None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:25.569599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4dc126-8961-4caa-a4c9-0693b15cca26 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:25.577131 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae0dbb-7a58-49bf-8c08-6a0cb9bf1044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:25.577773 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae0dbb-7a58-49bf-8c08-6a0cb9bf1044 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:25.578048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae0dbb-7a58-49bf-8c08-6a0cb9bf1044 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:25.578316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae0dbb-7a58-49bf-8c08-6a0cb9bf1044 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:25.578940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae0dbb-7a58-49bf-8c08-6a0cb9bf1044 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:25.580430 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ae0dbb-7a58-49bf-8c08-6a0cb9bf1044 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:25.587540 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 395/786] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Sep 8 15:55:25 2022] GET /identity/v3/services => generated 772 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:25.618256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:25.656388 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd040ce-d8c9-4f56-81b8-79da64689db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:25.657157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd040ce-d8c9-4f56-81b8-79da64689db3 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:25.657434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd040ce-d8c9-4f56-81b8-79da64689db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:25.657712 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd040ce-d8c9-4f56-81b8-79da64689db3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:25.666432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd040ce-d8c9-4f56-81b8-79da64689db3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:25.667963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd040ce-d8c9-4f56-81b8-79da64689db3 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-2bd040ce-d8c9-4f56-81b8-79da64689db3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:25.681244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:25.682284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:25.683246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:25.683246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:25.683246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:25.683246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:25.683246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:25.683246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:25.683246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:25.683246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:25.683246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 15:55:25.683246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:25.683246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 392/787] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:55:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:25.688886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae0dbb-7a58-49bf-8c08-6a0cb9bf1044 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:25.736537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62301ced-f826-4965-8924-cfee2c342d21 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:25.736864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62301ced-f826-4965-8924-cfee2c342d21 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:25.737093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62301ced-f826-4965-8924-cfee2c342d21 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:25.738715 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 393/788] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:25.753263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49309c-a36c-44a3-900b-d62c94d91998 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:25.753590 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49309c-a36c-44a3-900b-d62c94d91998 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:25.753810 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49309c-a36c-44a3-900b-d62c94d91998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:25.787157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-0a49309c-a36c-44a3-900b-d62c94d91998 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:25.788430 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0a49309c-a36c-44a3-900b-d62c94d91998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:25.801798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:25.824806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ae0dbb-7a58-49bf-8c08-6a0cb9bf1044 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:25.872747 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48a48a-dd8b-4b25-8ec5-bc1ecba33a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:25.873376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48a48a-dd8b-4b25-8ec5-bc1ecba33a61 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:25.873654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48a48a-dd8b-4b25-8ec5-bc1ecba33a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:25.873923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48a48a-dd8b-4b25-8ec5-bc1ecba33a61 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:25.874668 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48a48a-dd8b-4b25-8ec5-bc1ecba33a61 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:25.874972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48a48a-dd8b-4b25-8ec5-bc1ecba33a61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:25.876503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48a48a-dd8b-4b25-8ec5-bc1ecba33a61 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:25.897246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 394/789] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 15:55:25.897738 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 396/790] 10.222.0.27 () {56 vars in 1135 bytes} [Thu Sep 8 15:55:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:25.903613 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49309c-a36c-44a3-900b-d62c94d91998 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:26.038161 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49309c-a36c-44a3-900b-d62c94d91998 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:26.048138 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:26.074237 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddaa84a-5c61-47af-878c-65092952ebdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:26.075252 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaa84a-5c61-47af-878c-65092952ebdd None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:26.075752 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaa84a-5c61-47af-878c-65092952ebdd None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:26.076224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaa84a-5c61-47af-878c-65092952ebdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:26.087076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddaa84a-5c61-47af-878c-65092952ebdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:26.088837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddaa84a-5c61-47af-878c-65092952ebdd None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-2ddaa84a-5c61-47af-878c-65092952ebdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:26.104349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:26.105371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:26.106296 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:26.106296 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:26.106296 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:26.108509 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 395/791] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:26.116547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48a48a-dd8b-4b25-8ec5-bc1ecba33a61 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:26.193919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddaa84a-5c61-47af-878c-65092952ebdd None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:26.271178 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48a48a-dd8b-4b25-8ec5-bc1ecba33a61 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:26.336131 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fa8c3-99b8-463c-94b5-e90f7e291a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:26.338193 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fa8c3-99b8-463c-94b5-e90f7e291a09 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:26.339330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fa8c3-99b8-463c-94b5-e90f7e291a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:26.340203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fa8c3-99b8-463c-94b5-e90f7e291a09 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:26.341083 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fa8c3-99b8-463c-94b5-e90f7e291a09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:26.344700 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fa8c3-99b8-463c-94b5-e90f7e291a09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:26.346453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fa8c3-99b8-463c-94b5-e90f7e291a09 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:26.367532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddaa84a-5c61-47af-878c-65092952ebdd None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:26.375727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 397/792] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:26.380934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fa8c3-99b8-463c-94b5-e90f7e291a09 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:26.398403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:26.405703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa516a-3989-4f63-a85c-feb16a1c0bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:26.406706 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa516a-3989-4f63-a85c-feb16a1c0bde None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:26.407227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa516a-3989-4f63-a85c-feb16a1c0bde None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:26.407722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa516a-3989-4f63-a85c-feb16a1c0bde None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:26.408711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa516a-3989-4f63-a85c-feb16a1c0bde None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:26.410443 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa516a-3989-4f63-a85c-feb16a1c0bde None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-90fa516a-3989-4f63-a85c-feb16a1c0bde None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:26.420091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:26.421115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:26.421979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:26.421979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:26.421979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:26.421979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:26.421979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:26.421979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:26.421979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:26.421979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:26.421979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:26.421979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:26.421979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:26.421979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:26.421979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 15:55:26.421979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:26.424913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 396/793] 10.222.0.27 () {56 vars in 1129 bytes} [Thu Sep 8 15:55:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:26.430648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa516a-3989-4f63-a85c-feb16a1c0bde None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:26.480619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:26.576667 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fa8c3-99b8-463c-94b5-e90f7e291a09 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:26.626824 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa516a-3989-4f63-a85c-feb16a1c0bde None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:26.642049 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59c650-3b25-44fa-bc90-9f57d7e4e09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:26.643070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59c650-3b25-44fa-bc90-9f57d7e4e09a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:26.645869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59c650-3b25-44fa-bc90-9f57d7e4e09a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:26.646369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59c650-3b25-44fa-bc90-9f57d7e4e09a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:26.658983 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59c650-3b25-44fa-bc90-9f57d7e4e09a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:26.661523 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59c650-3b25-44fa-bc90-9f57d7e4e09a None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:26.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-9a59c650-3b25-44fa-bc90-9f57d7e4e09a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:26.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:26.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:26.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:26.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:26.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:26.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:26.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:26.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:26.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:26.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:26.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:26.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:26.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:26.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:26.688685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:26.689686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:26.689686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:26.689686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:26.689686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:26.689686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:26.689686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:26.689686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:26.689686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:26.696393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 398/794] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:26.698287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab8d2e-5a99-4573-9979-07a1b3792e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:26.699469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59c650-3b25-44fa-bc90-9f57d7e4e09a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:26.702243 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab8d2e-5a99-4573-9979-07a1b3792e4a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:26.702794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab8d2e-5a99-4573-9979-07a1b3792e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:26.704266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab8d2e-5a99-4573-9979-07a1b3792e4a None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:26.705227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab8d2e-5a99-4573-9979-07a1b3792e4a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:26.706025 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab8d2e-5a99-4573-9979-07a1b3792e4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:26.711405 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab8d2e-5a99-4573-9979-07a1b3792e4a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:26.723431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 397/795] 10.222.0.27 () {56 vars in 1124 bytes} [Thu Sep 8 15:55:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:26.728940 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab8d2e-5a99-4573-9979-07a1b3792e4a None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:26.839106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:26.861460 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:26.876535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab8d2e-5a99-4573-9979-07a1b3792e4a None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:26.890747 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59c650-3b25-44fa-bc90-9f57d7e4e09a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:26.923194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4123562-51b1-4421-bc94-1edb8da453ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:26.924270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4123562-51b1-4421-bc94-1edb8da453ef None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:26.924550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4123562-51b1-4421-bc94-1edb8da453ef None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:26.924815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4123562-51b1-4421-bc94-1edb8da453ef None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:26.925569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4123562-51b1-4421-bc94-1edb8da453ef None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:26.925991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4123562-51b1-4421-bc94-1edb8da453ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:26.927639 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4123562-51b1-4421-bc94-1edb8da453ef None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:26.933082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bbc605-2a52-4916-ae8e-b0d6a3fd506c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:26.933452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 398/796] 10.222.0.27 () {56 vars in 1124 bytes} [Thu Sep 8 15:55:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:26.934336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbc605-2a52-4916-ae8e-b0d6a3fd506c None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:26.934684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee9cca-9323-415a-b67c-2deaa8c21b35 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:26.934959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee9cca-9323-415a-b67c-2deaa8c21b35 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:26.935225 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee9cca-9323-415a-b67c-2deaa8c21b35 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:26.935425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbc605-2a52-4916-ae8e-b0d6a3fd506c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:26.935914 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bbc605-2a52-4916-ae8e-b0d6a3fd506c None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:26.936866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbc605-2a52-4916-ae8e-b0d6a3fd506c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:26.937456 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbc605-2a52-4916-ae8e-b0d6a3fd506c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:26.939152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bbc605-2a52-4916-ae8e-b0d6a3fd506c None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:26.940690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 399/797] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:26.941783 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee9cca-9323-415a-b67c-2deaa8c21b35 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:26.954537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 399/798] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:26.956375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b5bb9-e19d-4602-a421-02a7d0eb60c4 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:26.956865 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b5bb9-e19d-4602-a421-02a7d0eb60c4 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:26.957349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b5bb9-e19d-4602-a421-02a7d0eb60c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:26.976259 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-268b5bb9-e19d-4602-a421-02a7d0eb60c4 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:26.977744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-268b5bb9-e19d-4602-a421-02a7d0eb60c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:27.050429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 400/799] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:27.051683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b5bb9-e19d-4602-a421-02a7d0eb60c4 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:27.081703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee9cca-9323-415a-b67c-2deaa8c21b35 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:27.147413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:27.153653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb34a3-6516-4f63-8c2a-1489a4905a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:27.154286 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb34a3-6516-4f63-8c2a-1489a4905a4a None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:27.154567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb34a3-6516-4f63-8c2a-1489a4905a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:27.154836 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb34a3-6516-4f63-8c2a-1489a4905a4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:27.155550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb34a3-6516-4f63-8c2a-1489a4905a4a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:27.156933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb34a3-6516-4f63-8c2a-1489a4905a4a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:27.214202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b5bb9-e19d-4602-a421-02a7d0eb60c4 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:27.230634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 400/800] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 15:55:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:55:27.257440 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96193160-ffa9-4728-a3ea-cb2808def55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:27.258481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96193160-ffa9-4728-a3ea-cb2808def55a None admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:27.259017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96193160-ffa9-4728-a3ea-cb2808def55a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:27.259509 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96193160-ffa9-4728-a3ea-cb2808def55a None admin] PATH_INFO: `/v3/projects/a825a0b1238d4382995da440944f453b/users/de54687547a14ab89cc41a1fabaab400/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:27.261929 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96193160-ffa9-4728-a3ea-cb2808def55a None admin] RBAC: Authorizing `identity:create_grant(project_id=a825a0b1238d4382995da440944f453b, user_id=de54687547a14ab89cc41a1fabaab400, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:27.263779 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96193160-ffa9-4728-a3ea-cb2808def55a None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:27.274970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed9c57-092c-47fc-92be-175134d34d5f None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:27.274970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed9c57-092c-47fc-92be-175134d34d5f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:27.274970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed9c57-092c-47fc-92be-175134d34d5f None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:27.274970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 401/801] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:27.279343 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5002a-c5ae-4380-b442-55fed960a3dd None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:27.279931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5002a-c5ae-4380-b442-55fed960a3dd None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:27.280421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5002a-c5ae-4380-b442-55fed960a3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:27.293859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 401/802] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:55:27 2022] PUT /identity/v3/projects/a825a0b1238d4382995da440944f453b/users/de54687547a14ab89cc41a1fabaab400/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:55:27.311052 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-76f5002a-c5ae-4380-b442-55fed960a3dd None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:27.312866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-76f5002a-c5ae-4380-b442-55fed960a3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:27.316075 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:27.418605 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 402/803] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:27.425424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96193160-ffa9-4728-a3ea-cb2808def55a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:27.537781 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96193160-ffa9-4728-a3ea-cb2808def55a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:27.548197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:27.578137 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96074f8c-84b1-4e79-a686-62bdb725818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:27.578788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96074f8c-84b1-4e79-a686-62bdb725818d None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:27.579065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96074f8c-84b1-4e79-a686-62bdb725818d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:27.579338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96074f8c-84b1-4e79-a686-62bdb725818d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:27.590387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96074f8c-84b1-4e79-a686-62bdb725818d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-96074f8c-84b1-4e79-a686-62bdb725818d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:27.595424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:27.596484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:27.597352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:27.597352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:27.597352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:27.597352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:27.597352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:27.597352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:27.597352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:27.597352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:27.597352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:27.597352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:27.597352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:27.597352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:27.601639 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 402/804] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:55:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:27.603099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5002a-c5ae-4380-b442-55fed960a3dd None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:27.728568 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5002a-c5ae-4380-b442-55fed960a3dd None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:27.753931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96074f8c-84b1-4e79-a686-62bdb725818d None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:27.763223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d1d00-b08c-4271-94dd-29a11884530f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:27.763881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d1d00-b08c-4271-94dd-29a11884530f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:27.764291 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d1d00-b08c-4271-94dd-29a11884530f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:27.764555 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d1d00-b08c-4271-94dd-29a11884530f None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:27.765224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d1d00-b08c-4271-94dd-29a11884530f None admin] RBAC: Adding query filter params (name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:27.765526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d1d00-b08c-4271-94dd-29a11884530f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:27.766916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d1d00-b08c-4271-94dd-29a11884530f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:27.775802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 403/805] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:55:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:27.779803 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:27.783320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d1d00-b08c-4271-94dd-29a11884530f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:27.925467 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96074f8c-84b1-4e79-a686-62bdb725818d None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:27.963323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0b96d-e082-4337-a1a0-128d7c5798cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:27.963885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0b96d-e082-4337-a1a0-128d7c5798cd None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:27.964158 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0b96d-e082-4337-a1a0-128d7c5798cd None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:27.964433 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0b96d-e082-4337-a1a0-128d7c5798cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:27.977443 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0b96d-e082-4337-a1a0-128d7c5798cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-71b0b96d-e082-4337-a1a0-128d7c5798cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:27.981715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:27.982651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:27.983532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:27.983532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:27.983532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:27.983532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:27.983532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:27.983532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:27.983532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:27.983532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:27.983532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:27.983532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:27.983532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:55:27.983532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:27.983532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 403/806] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:55:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:27.990900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0b96d-e082-4337-a1a0-128d7c5798cd None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:27.998451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d1d00-b08c-4271-94dd-29a11884530f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:28.042200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c025340-782b-458c-9d30-2fb07c542390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:28.043423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c025340-782b-458c-9d30-2fb07c542390 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:28.044580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:28.045412 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c025340-782b-458c-9d30-2fb07c542390 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:28.046551 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c025340-782b-458c-9d30-2fb07c542390 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:28.066356 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c025340-782b-458c-9d30-2fb07c542390 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:28.068926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c025340-782b-458c-9d30-2fb07c542390 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-2c025340-782b-458c-9d30-2fb07c542390 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:28.095311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:28.096423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:28.097328 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:28.097328 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:28.097328 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:28.099334 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 404/807] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:28.104426 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c025340-782b-458c-9d30-2fb07c542390 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:28.186577 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0b96d-e082-4337-a1a0-128d7c5798cd None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:28.203362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:28.229646 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8517254-dd1f-4ec5-80a5-3c9edb9c69ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:28.230264 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8517254-dd1f-4ec5-80a5-3c9edb9c69ff None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:28.230537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8517254-dd1f-4ec5-80a5-3c9edb9c69ff None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:28.230805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8517254-dd1f-4ec5-80a5-3c9edb9c69ff None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:28.231504 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8517254-dd1f-4ec5-80a5-3c9edb9c69ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:28.231869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8517254-dd1f-4ec5-80a5-3c9edb9c69ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:28.233183 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8517254-dd1f-4ec5-80a5-3c9edb9c69ff None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:28.243900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 404/808] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:55:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:28.245161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48ff3d-65f3-4943-8588-173bcc94c4be None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:28.245440 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48ff3d-65f3-4943-8588-173bcc94c4be None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:28.245708 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48ff3d-65f3-4943-8588-173bcc94c4be None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:28.255879 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 405/809] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:28.255879 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48ff3d-65f3-4943-8588-173bcc94c4be None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:28.272363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c025340-782b-458c-9d30-2fb07c542390 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:28.311009 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de079479-8460-4d64-852b-dfa80ab229f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:28.311674 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de079479-8460-4d64-852b-dfa80ab229f8 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:28.311987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de079479-8460-4d64-852b-dfa80ab229f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:28.312256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de079479-8460-4d64-852b-dfa80ab229f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:28.313043 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de079479-8460-4d64-852b-dfa80ab229f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:28.313347 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de079479-8460-4d64-852b-dfa80ab229f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:28.314940 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de079479-8460-4d64-852b-dfa80ab229f8 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:28.334298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 405/810] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:28.334298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0bf65-5376-4066-89c2-2e3994185c72 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:28.334298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0bf65-5376-4066-89c2-2e3994185c72 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:28.334298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0bf65-5376-4066-89c2-2e3994185c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:28.359038 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-4bd0bf65-5376-4066-89c2-2e3994185c72 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:28.360740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-4bd0bf65-5376-4066-89c2-2e3994185c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:28.378163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:28.391946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48ff3d-65f3-4943-8588-173bcc94c4be None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:28.434448 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4050fb-da8a-4f9a-be90-078c3e94b0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:28.435532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4050fb-da8a-4f9a-be90-078c3e94b0a6 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:28.436063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4050fb-da8a-4f9a-be90-078c3e94b0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:28.436602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4050fb-da8a-4f9a-be90-078c3e94b0a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:28.447692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4050fb-da8a-4f9a-be90-078c3e94b0a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:28.449371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4050fb-da8a-4f9a-be90-078c3e94b0a6 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:28.467478 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 406/811] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:28.467478 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0bf65-5376-4066-89c2-2e3994185c72 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-5f4050fb-da8a-4f9a-be90-078c3e94b0a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:28.470989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:28.472047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:28.472962 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:28.472962 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:28.472962 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:28.479416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 406/812] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:28.480487 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4050fb-da8a-4f9a-be90-078c3e94b0a6 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:28.519561 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:28.588680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0bf65-5376-4066-89c2-2e3994185c72 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:28.628081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c98c6a-a680-4182-9733-b568557d1aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:28.628966 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c98c6a-a680-4182-9733-b568557d1aea None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:28.629259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c98c6a-a680-4182-9733-b568557d1aea None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:28.629899 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c98c6a-a680-4182-9733-b568557d1aea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:28.637640 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c98c6a-a680-4182-9733-b568557d1aea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:28.639397 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c98c6a-a680-4182-9733-b568557d1aea None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-58c98c6a-a680-4182-9733-b568557d1aea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:28.655648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:28.656661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:28.657606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:28.657606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:28.657606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:28.657606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:28.657606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:28.657606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:28.657606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:28.657606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:28.657606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 15:55:28.657606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:28.658929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 407/813] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Sep 8 15:55:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:28.664068 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c98c6a-a680-4182-9733-b568557d1aea None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:28.733617 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4050fb-da8a-4f9a-be90-078c3e94b0a6 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:28.779187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c98c6a-a680-4182-9733-b568557d1aea None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:28.781175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:28.814988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e5050-2358-45ad-ad18-f8d9898829f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:28.815720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e5050-2358-45ad-ad18-f8d9898829f6 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:28.816006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e5050-2358-45ad-ad18-f8d9898829f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:28.816275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e5050-2358-45ad-ad18-f8d9898829f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:28.816931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e5050-2358-45ad-ad18-f8d9898829f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:28.817231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e5050-2358-45ad-ad18-f8d9898829f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:28.818694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e5050-2358-45ad-ad18-f8d9898829f6 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:28.820465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d45b8-fb0b-4fc0-8f8f-4d9b59bb6ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:28.821743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d45b8-fb0b-4fc0-8f8f-4d9b59bb6ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:28.822223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d45b8-fb0b-4fc0-8f8f-4d9b59bb6ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:28.822692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0d45b8-fb0b-4fc0-8f8f-4d9b59bb6ea6 None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:28.823664 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d45b8-fb0b-4fc0-8f8f-4d9b59bb6ea6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:28.824169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d45b8-fb0b-4fc0-8f8f-4d9b59bb6ea6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:28.825803 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0d45b8-fb0b-4fc0-8f8f-4d9b59bb6ea6 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:28.832748 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 407/814] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:28.835782 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e5050-2358-45ad-ad18-f8d9898829f6 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:28.839102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:28.852883 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 408/815] 10.222.0.27 () {56 vars in 1149 bytes} [Thu Sep 8 15:55:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:28.857636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d45b8-fb0b-4fc0-8f8f-4d9b59bb6ea6 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:28.946070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e5050-2358-45ad-ad18-f8d9898829f6 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:28.969827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0d45b8-fb0b-4fc0-8f8f-4d9b59bb6ea6 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:28.981061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4f185-eba2-4a4f-ae12-6c29bdcc8901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:28.981707 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4f185-eba2-4a4f-ae12-6c29bdcc8901 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:28.981987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4f185-eba2-4a4f-ae12-6c29bdcc8901 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:28.982256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4f185-eba2-4a4f-ae12-6c29bdcc8901 None admin] PATH_INFO: `/v3/services/network` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:28.982859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4f185-eba2-4a4f-ae12-6c29bdcc8901 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:28.984292 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4f185-eba2-4a4f-ae12-6c29bdcc8901 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-b1c4f185-eba2-4a4f-ae12-6c29bdcc8901 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:28.993337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:28.994321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:28.995167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:28.995167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:28.995167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:28.995167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:28.995167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:28.995167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:28.995167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:28.995167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:28.995167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:28.995167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:28.995167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:28.995167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:28.995167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 15:55:28.995167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:28.999338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 408/816] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:28 2022] GET /identity/v3/services/network => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:29.000273 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4f185-eba2-4a4f-ae12-6c29bdcc8901 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:29.013133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f0fae-0e7e-4fab-95b7-2b7019a65c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:29.014205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f0fae-0e7e-4fab-95b7-2b7019a65c64 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:29.014717 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f0fae-0e7e-4fab-95b7-2b7019a65c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:29.015218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f0fae-0e7e-4fab-95b7-2b7019a65c64 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:29.023687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f0fae-0e7e-4fab-95b7-2b7019a65c64 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:29.025363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0f0fae-0e7e-4fab-95b7-2b7019a65c64 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-df0f0fae-0e7e-4fab-95b7-2b7019a65c64 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:29.038583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:29.039631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:29.040610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:29.040610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:29.040610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:29.040610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:29.040610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:29.040610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:29.040610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:29.040610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:29.040610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:55:29.040610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:29.043217 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 409/817] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Sep 8 15:55:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:29.048975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f0fae-0e7e-4fab-95b7-2b7019a65c64 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:29.070687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:29.077169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:29.107025 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4f185-eba2-4a4f-ae12-6c29bdcc8901 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:29.141148 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d92cf0-5a30-4704-84d8-416bb20e69f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:29.141793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d92cf0-5a30-4704-84d8-416bb20e69f9 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:29.142073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d92cf0-5a30-4704-84d8-416bb20e69f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:29.142345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d92cf0-5a30-4704-84d8-416bb20e69f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:29.153553 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d92cf0-5a30-4704-84d8-416bb20e69f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:29.154671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d92cf0-5a30-4704-84d8-416bb20e69f9 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:29.166724 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f0fae-0e7e-4fab-95b7-2b7019a65c64 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-a4d92cf0-5a30-4704-84d8-416bb20e69f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:29.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:29.176122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:29.177018 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:29.177018 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:29.177018 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:29.186467 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 409/818] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:29.186837 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d92cf0-5a30-4704-84d8-416bb20e69f9 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:29.208448 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a94ef2-dc22-4a16-a4d2-f76ea09daca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:29.209467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a94ef2-dc22-4a16-a4d2-f76ea09daca4 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:29.209983 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a94ef2-dc22-4a16-a4d2-f76ea09daca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:29.210479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a94ef2-dc22-4a16-a4d2-f76ea09daca4 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:29.211410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a94ef2-dc22-4a16-a4d2-f76ea09daca4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:29.212281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a94ef2-dc22-4a16-a4d2-f76ea09daca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:29.213907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a94ef2-dc22-4a16-a4d2-f76ea09daca4 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:29.219350 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 410/819] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Sep 8 15:55:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:29.220393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a94ef2-dc22-4a16-a4d2-f76ea09daca4 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:29.300954 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d92cf0-5a30-4704-84d8-416bb20e69f9 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:29.318301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:29.323961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:29.344005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2d5c3-fb3b-492b-98e9-5df2b9f44a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:29.344669 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a94ef2-dc22-4a16-a4d2-f76ea09daca4 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:29.345261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2d5c3-fb3b-492b-98e9-5df2b9f44a57 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:29.345802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2d5c3-fb3b-492b-98e9-5df2b9f44a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:29.346496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2d5c3-fb3b-492b-98e9-5df2b9f44a57 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:29.348238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2d5c3-fb3b-492b-98e9-5df2b9f44a57 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:29.348755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2d5c3-fb3b-492b-98e9-5df2b9f44a57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:29.350360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2d5c3-fb3b-492b-98e9-5df2b9f44a57 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:29.373635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 410/820] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Sep 8 15:55:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:29.373635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2d5c3-fb3b-492b-98e9-5df2b9f44a57 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:29.391427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4dbc0-2731-458c-9e9a-072cfd5c982e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:29.392500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4dbc0-2731-458c-9e9a-072cfd5c982e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:29.393115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4dbc0-2731-458c-9e9a-072cfd5c982e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:29.393582 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4dbc0-2731-458c-9e9a-072cfd5c982e None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:29.394421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4dbc0-2731-458c-9e9a-072cfd5c982e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:29.395005 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4dbc0-2731-458c-9e9a-072cfd5c982e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:29.396840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4dbc0-2731-458c-9e9a-072cfd5c982e None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:29.411462 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 411/821] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:29.412653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4dbc0-2731-458c-9e9a-072cfd5c982e None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:29.481900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2d5c3-fb3b-492b-98e9-5df2b9f44a57 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:29.518897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3a4b6-2b61-4246-b88f-cbda3869c8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:29.519567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3a4b6-2b61-4246-b88f-cbda3869c8ef None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:29.519855 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3a4b6-2b61-4246-b88f-cbda3869c8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:29.520239 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3a4b6-2b61-4246-b88f-cbda3869c8ef None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:29.520844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3a4b6-2b61-4246-b88f-cbda3869c8ef None admin] RBAC: Adding query filter params (type=network) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:29.521148 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3a4b6-2b61-4246-b88f-cbda3869c8ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:29.522647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3a4b6-2b61-4246-b88f-cbda3869c8ef None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:29.525181 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4dbc0-2731-458c-9e9a-072cfd5c982e None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:29.533648 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 411/822] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Sep 8 15:55:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:29.535244 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3a4b6-2b61-4246-b88f-cbda3869c8ef None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:29.566992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0f6c8-b2f8-44b4-a6dd-99801ee448df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:29.567976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0f6c8-b2f8-44b4-a6dd-99801ee448df None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:29.568471 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0f6c8-b2f8-44b4-a6dd-99801ee448df None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:29.568932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0f6c8-b2f8-44b4-a6dd-99801ee448df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:29.582642 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:29.585870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:29.645026 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3a4b6-2b61-4246-b88f-cbda3869c8ef None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:29.647361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0f6c8-b2f8-44b4-a6dd-99801ee448df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:29.648881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0f6c8-b2f8-44b4-a6dd-99801ee448df None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:29.689481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b1f13-4c71-46f4-af27-f261d1bd6ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:29.690158 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b1f13-4c71-46f4-af27-f261d1bd6ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:29.690435 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b1f13-4c71-46f4-af27-f261d1bd6ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:29.690704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454b1f13-4c71-46f4-af27-f261d1bd6ae9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:29.698454 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b1f13-4c71-46f4-af27-f261d1bd6ae9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:29.700147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454b1f13-4c71-46f4-af27-f261d1bd6ae9 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:29.712476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-454b1f13-4c71-46f4-af27-f261d1bd6ae9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:29.712476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:29.712476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:29.712476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:29.712476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:29.712476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:29.712476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:29.712476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:29.712476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:29.712476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:29.712476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:29.712476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:29.712476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:29.712476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:29.712476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:29.713584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:29.714472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:29.714472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:29.714472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:29.714472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:29.714472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:29.714472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:29.714472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:29.714472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:29.714472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 412/823] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:29.714472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b1f13-4c71-46f4-af27-f261d1bd6ae9 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:29.775914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 412/824] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:55:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:55:29.775914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5ab4f-d0de-4a56-9006-414552780f7c None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:29.775914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5ab4f-d0de-4a56-9006-414552780f7c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:29.775914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f5ab4f-d0de-4a56-9006-414552780f7c None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:29.775914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 413/825] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:29.775914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5ab4f-d0de-4a56-9006-414552780f7c None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:29.824628 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:29.828901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:29.832910 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454b1f13-4c71-46f4-af27-f261d1bd6ae9 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:29.869068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282bdee-b64d-4514-89be-fdcbc42a7b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:29.869721 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282bdee-b64d-4514-89be-fdcbc42a7b09 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:29.869993 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282bdee-b64d-4514-89be-fdcbc42a7b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:29.870260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282bdee-b64d-4514-89be-fdcbc42a7b09 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:29.870916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282bdee-b64d-4514-89be-fdcbc42a7b09 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:29.872309 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282bdee-b64d-4514-89be-fdcbc42a7b09 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:29.882964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f5ab4f-d0de-4a56-9006-414552780f7c None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:29.894648 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 413/826] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 15:55:29 2022] POST /identity/v3/endpoints => generated 337 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:55:29.895717 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282bdee-b64d-4514-89be-fdcbc42a7b09 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:29.928412 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a479c55-9d65-4f23-a9f2-16aaac6e377e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:29.929226 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a479c55-9d65-4f23-a9f2-16aaac6e377e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:29.929531 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a479c55-9d65-4f23-a9f2-16aaac6e377e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:29.929798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a479c55-9d65-4f23-a9f2-16aaac6e377e None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:29.930573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a479c55-9d65-4f23-a9f2-16aaac6e377e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:29.930881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a479c55-9d65-4f23-a9f2-16aaac6e377e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:29.932899 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a479c55-9d65-4f23-a9f2-16aaac6e377e None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:29.946337 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 414/827] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:29.947840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b34f1-7f18-4880-9028-20a0dd060fe8 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:29.948125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b34f1-7f18-4880-9028-20a0dd060fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:29.948395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129b34f1-7f18-4880-9028-20a0dd060fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:29.969103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-129b34f1-7f18-4880-9028-20a0dd060fe8 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:29.970352 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-129b34f1-7f18-4880-9028-20a0dd060fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:30.022084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282bdee-b64d-4514-89be-fdcbc42a7b09 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:30.049060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 415/828] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:30.049060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403b476-653b-47e6-ab38-53b6b9d4e1fb None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:30.049060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403b476-653b-47e6-ab38-53b6b9d4e1fb None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:30.049060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403b476-653b-47e6-ab38-53b6b9d4e1fb None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:30.049060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 416/829] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:30.049060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403b476-653b-47e6-ab38-53b6b9d4e1fb None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:30.069520 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f7d06-e6fe-46c6-882c-baf2f82222a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:30.070185 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f7d06-e6fe-46c6-882c-baf2f82222a2 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:30.070462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f7d06-e6fe-46c6-882c-baf2f82222a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:30.070730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f7d06-e6fe-46c6-882c-baf2f82222a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:30.086331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f7d06-e6fe-46c6-882c-baf2f82222a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:30.088156 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f7d06-e6fe-46c6-882c-baf2f82222a2 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:30.098332 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-888f7d06-e6fe-46c6-882c-baf2f82222a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:30.107416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:30.108429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:30.109313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:30.109313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:30.109313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:30.109313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 414/830] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:30.110804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0891e12-6ad2-4958-b4da-d13a4a11657e None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:30.110804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0891e12-6ad2-4958-b4da-d13a4a11657e None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:30.110804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0891e12-6ad2-4958-b4da-d13a4a11657e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:30.129549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-f0891e12-6ad2-4958-b4da-d13a4a11657e None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:30.130740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f0891e12-6ad2-4958-b4da-d13a4a11657e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:30.138663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:30.194577 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403b476-653b-47e6-ab38-53b6b9d4e1fb None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:30.223281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 415/831] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:30.224893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0891e12-6ad2-4958-b4da-d13a4a11657e None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:30.239458 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad0d99-375d-41bb-88b9-33b3b6efb3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:30.240257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad0d99-375d-41bb-88b9-33b3b6efb3d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:30.240746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad0d99-375d-41bb-88b9-33b3b6efb3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:30.241096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad0d99-375d-41bb-88b9-33b3b6efb3d6 None admin] PATH_INFO: `/v3/projects/a825a0b1238d4382995da440944f453b/users/b4324e4c04e343c98d9bb216c542fcba/roles/b256252b45e541a491376a134f502183` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:30.260340 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad0d99-375d-41bb-88b9-33b3b6efb3d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a825a0b1238d4382995da440944f453b, user_id=b4324e4c04e343c98d9bb216c542fcba, role_id=b256252b45e541a491376a134f502183)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:30.262707 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad0d99-375d-41bb-88b9-33b3b6efb3d6 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:30.279205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 417/832] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:55:30 2022] PUT /identity/v3/projects/a825a0b1238d4382995da440944f453b/users/b4324e4c04e343c98d9bb216c542fcba/roles/b256252b45e541a491376a134f502183 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:55:30.334433 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad0d99-375d-41bb-88b9-33b3b6efb3d6 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:30.341206 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0891e12-6ad2-4958-b4da-d13a4a11657e None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:30.397337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c9819-aa6f-409a-bb95-d322071007bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:30.398005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c9819-aa6f-409a-bb95-d322071007bf None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:30.398404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c9819-aa6f-409a-bb95-d322071007bf None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:30.398669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c9819-aa6f-409a-bb95-d322071007bf None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:30.399387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c9819-aa6f-409a-bb95-d322071007bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:30.399736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c9819-aa6f-409a-bb95-d322071007bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:30.401158 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c9819-aa6f-409a-bb95-d322071007bf None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:30.416808 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:30.417126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 416/833] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:30.422863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c9819-aa6f-409a-bb95-d322071007bf None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:30.435126 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:30.459287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad0d99-375d-41bb-88b9-33b3b6efb3d6 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:30.498510 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5c264-7f55-401b-a416-808602648671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:30.499717 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5c264-7f55-401b-a416-808602648671 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:30.500171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5c264-7f55-401b-a416-808602648671 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:30.500670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5c264-7f55-401b-a416-808602648671 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:30.512481 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5c264-7f55-401b-a416-808602648671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-8fe5c264-7f55-401b-a416-808602648671 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:30.517535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:30.518506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:30.519787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:30.519787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:30.519787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:30.519787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:30.519787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:30.519787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:30.519787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:30.519787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:30.519787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:30.519787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:30.519787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:55:30.519787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:30.519787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 418/834] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:55:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:30.519787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5c264-7f55-401b-a416-808602648671 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:30.542959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c9819-aa6f-409a-bb95-d322071007bf None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:30.577756 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376700c-0993-4c58-92a5-b085abac33f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:30.578389 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376700c-0993-4c58-92a5-b085abac33f9 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:30.578666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376700c-0993-4c58-92a5-b085abac33f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:30.578931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8376700c-0993-4c58-92a5-b085abac33f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:30.586322 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376700c-0993-4c58-92a5-b085abac33f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:30.588124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8376700c-0993-4c58-92a5-b085abac33f9 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:30.600378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-8376700c-0993-4c58-92a5-b085abac33f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:30.600378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:30.600378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:30.600378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:30.600378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:30.600378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:30.600378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:30.600378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:30.600378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:30.600378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:30.600378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:30.600378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:30.600378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:30.600378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:30.600378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:30.601295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:30.602395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:30.602395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:30.602395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:30.602395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:30.602395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:30.602395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:30.602395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:30.602395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:30.602395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 417/835] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:30.606918 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376700c-0993-4c58-92a5-b085abac33f9 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:30.627804 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5c264-7f55-401b-a416-808602648671 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:30.663221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95a08a-479b-4498-bf3b-0d5906fcee91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:30.663986 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95a08a-479b-4498-bf3b-0d5906fcee91 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:30.664261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95a08a-479b-4498-bf3b-0d5906fcee91 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:30.664528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95a08a-479b-4498-bf3b-0d5906fcee91 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:30.665110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95a08a-479b-4498-bf3b-0d5906fcee91 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:30.666488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc95a08a-479b-4498-bf3b-0d5906fcee91 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:30.670171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-cc95a08a-479b-4498-bf3b-0d5906fcee91 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:30.676787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:30.677802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:30.680565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:30.680565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:30.680565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:30.680565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:30.680565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:30.680565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:30.680565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:30.680565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:30.680565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:30.680565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:30.680565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:30.680565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:30.680565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 15:55:30.680565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:30.680565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 419/836] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:55:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:30.681405 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95a08a-479b-4498-bf3b-0d5906fcee91 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:30.683785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:30.724311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8376700c-0993-4c58-92a5-b085abac33f9 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:30.760243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc85d1c-ecbd-4e9c-bee4-9065943da2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:30.760851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc85d1c-ecbd-4e9c-bee4-9065943da2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:30.761122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc85d1c-ecbd-4e9c-bee4-9065943da2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:30.761386 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc85d1c-ecbd-4e9c-bee4-9065943da2a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:30.762054 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc85d1c-ecbd-4e9c-bee4-9065943da2a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:30.762348 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc85d1c-ecbd-4e9c-bee4-9065943da2a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:30.763744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc85d1c-ecbd-4e9c-bee4-9065943da2a5 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:30.772380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 418/837] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:55:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:30.773021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc85d1c-ecbd-4e9c-bee4-9065943da2a5 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:30.792370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95a08a-479b-4498-bf3b-0d5906fcee91 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:30.826717 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c21951-0751-4109-be12-b093d34f92ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:30.827390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c21951-0751-4109-be12-b093d34f92ca None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:30.827750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c21951-0751-4109-be12-b093d34f92ca None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:30.828080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c21951-0751-4109-be12-b093d34f92ca None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:30.828840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c21951-0751-4109-be12-b093d34f92ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:30.829199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c21951-0751-4109-be12-b093d34f92ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:30.830860 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c21951-0751-4109-be12-b093d34f92ca None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:30.848151 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 420/838] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:30.848151 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c21951-0751-4109-be12-b093d34f92ca None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:30.887479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc85d1c-ecbd-4e9c-bee4-9065943da2a5 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:30.909437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:30.912074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:30.924301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0680e-b19d-443c-9846-9979c18ba6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:30.924909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0680e-b19d-443c-9846-9979c18ba6cc None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:30.925179 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0680e-b19d-443c-9846-9979c18ba6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:30.925444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0680e-b19d-443c-9846-9979c18ba6cc None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:30.926066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0680e-b19d-443c-9846-9979c18ba6cc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:30.926360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0680e-b19d-443c-9846-9979c18ba6cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:30.927750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0680e-b19d-443c-9846-9979c18ba6cc None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:30.932416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 419/839] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:55:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:30.933005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0680e-b19d-443c-9846-9979c18ba6cc None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:30.957276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c21951-0751-4109-be12-b093d34f92ca None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:30.991783 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fca078-8070-4ac1-ad4a-497af72cc154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:30.992402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fca078-8070-4ac1-ad4a-497af72cc154 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:30.992689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fca078-8070-4ac1-ad4a-497af72cc154 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:30.993205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fca078-8070-4ac1-ad4a-497af72cc154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:31.003469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fca078-8070-4ac1-ad4a-497af72cc154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:31.004945 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fca078-8070-4ac1-ad4a-497af72cc154 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-72fca078-8070-4ac1-ad4a-497af72cc154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:31.019918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:31.020951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:31.021856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:31.021856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:31.021856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:31.021856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 421/840] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:31.021856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fca078-8070-4ac1-ad4a-497af72cc154 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:31.046985 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0680e-b19d-443c-9846-9979c18ba6cc None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:31.082489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a9ba6-709e-4112-9a72-849cedc8270b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:31.083127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a9ba6-709e-4112-9a72-849cedc8270b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:31.083583 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a9ba6-709e-4112-9a72-849cedc8270b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:31.083820 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a9ba6-709e-4112-9a72-849cedc8270b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:31.084542 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a9ba6-709e-4112-9a72-849cedc8270b None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=6e9209e77ce44b128e25a8ddcf78f8b5) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:31.084849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a9ba6-709e-4112-9a72-849cedc8270b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:31.086264 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a9ba6-709e-4112-9a72-849cedc8270b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:31.103481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 420/841] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:55:30 2022] GET /identity/v3/role_assignments?user.id=6e9209e77ce44b128e25a8ddcf78f8b5&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:31.103481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a9ba6-709e-4112-9a72-849cedc8270b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:31.146859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fca078-8070-4ac1-ad4a-497af72cc154 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:31.155605 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:31.167175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:31.199428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed028745-a17e-4403-977f-8edd77e98f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:31.199737 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed028745-a17e-4403-977f-8edd77e98f5f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:31.199835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed028745-a17e-4403-977f-8edd77e98f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:31.200099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed028745-a17e-4403-977f-8edd77e98f5f None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:31.200759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed028745-a17e-4403-977f-8edd77e98f5f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:31.201059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed028745-a17e-4403-977f-8edd77e98f5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:31.202524 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed028745-a17e-4403-977f-8edd77e98f5f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:31.208234 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 422/842] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:55:31 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:31.231931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a9ba6-709e-4112-9a72-849cedc8270b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:31.273850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800a86a-1ec7-47e7-8ef7-a77088742893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:31.274829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800a86a-1ec7-47e7-8ef7-a77088742893 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:31.275267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800a86a-1ec7-47e7-8ef7-a77088742893 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:31.275772 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800a86a-1ec7-47e7-8ef7-a77088742893 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:31.276618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800a86a-1ec7-47e7-8ef7-a77088742893 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:31.277111 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800a86a-1ec7-47e7-8ef7-a77088742893 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:31.278725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800a86a-1ec7-47e7-8ef7-a77088742893 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:31.294223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 421/843] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:31.300546 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed028745-a17e-4403-977f-8edd77e98f5f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:31.417574 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed028745-a17e-4403-977f-8edd77e98f5f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:31.452972 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674820e-813e-4204-aa98-c4144f2c6f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:31.453938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674820e-813e-4204-aa98-c4144f2c6f04 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:31.454348 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674820e-813e-4204-aa98-c4144f2c6f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:31.454759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a674820e-813e-4204-aa98-c4144f2c6f04 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:31.464153 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674820e-813e-4204-aa98-c4144f2c6f04 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:31.465959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a674820e-813e-4204-aa98-c4144f2c6f04 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-a674820e-813e-4204-aa98-c4144f2c6f04 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:31.478128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:31.479290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:31.480501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:31.480501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:31.480501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:31.480501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:31.480501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:31.480501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:31.480501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:31.480501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:31.480501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 15:55:31.480501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:31.482724 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 423/844] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:55:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:31.492448 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800a86a-1ec7-47e7-8ef7-a77088742893 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:31.595984 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800a86a-1ec7-47e7-8ef7-a77088742893 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:31.605903 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:31.631563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1780117-f0ed-4e60-99de-293b9ffdb431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:31.632173 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1780117-f0ed-4e60-99de-293b9ffdb431 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:31.632447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1780117-f0ed-4e60-99de-293b9ffdb431 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:31.632766 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1780117-f0ed-4e60-99de-293b9ffdb431 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:31.633502 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1780117-f0ed-4e60-99de-293b9ffdb431 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:31.633799 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1780117-f0ed-4e60-99de-293b9ffdb431 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:31.635224 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1780117-f0ed-4e60-99de-293b9ffdb431 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:31.652984 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 422/845] 10.222.0.27 () {56 vars in 1135 bytes} [Thu Sep 8 15:55:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:31.658720 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674820e-813e-4204-aa98-c4144f2c6f04 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:31.824936 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:31.884913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a674820e-813e-4204-aa98-c4144f2c6f04 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:31.920093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1687e-d977-4d26-8f04-f033985ae740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:31.921064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1687e-d977-4d26-8f04-f033985ae740 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:31.921528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1687e-d977-4d26-8f04-f033985ae740 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:31.921981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1687e-d977-4d26-8f04-f033985ae740 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:31.990893 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1687e-d977-4d26-8f04-f033985ae740 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:31.992758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1687e-d977-4d26-8f04-f033985ae740 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:32.033853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:32.093736 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 424/846] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:55:31 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:55:32.099728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1780117-f0ed-4e60-99de-293b9ffdb431 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:32.208168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1780117-f0ed-4e60-99de-293b9ffdb431 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:32.242984 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b939b-0720-48e8-b59f-d5cace9ee3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:32.243652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b939b-0720-48e8-b59f-d5cace9ee3eb None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:32.243963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b939b-0720-48e8-b59f-d5cace9ee3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:32.244228 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178b939b-0720-48e8-b59f-d5cace9ee3eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:32.254393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b939b-0720-48e8-b59f-d5cace9ee3eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:32.255857 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178b939b-0720-48e8-b59f-d5cace9ee3eb None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-178b939b-0720-48e8-b59f-d5cace9ee3eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:32.271031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:32.272011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:32.272901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:32.272901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:32.272901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:32.272901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 423/847] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:32.280543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1687e-d977-4d26-8f04-f033985ae740 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:32.381979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1687e-d977-4d26-8f04-f033985ae740 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:32.419248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa688de-e425-4f23-a074-cc8c8f8f7dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:32.419888 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa688de-e425-4f23-a074-cc8c8f8f7dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:32.420153 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa688de-e425-4f23-a074-cc8c8f8f7dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:32.420412 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa688de-e425-4f23-a074-cc8c8f8f7dd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:32.421058 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa688de-e425-4f23-a074-cc8c8f8f7dd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:32.421352 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa688de-e425-4f23-a074-cc8c8f8f7dd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:32.422719 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa688de-e425-4f23-a074-cc8c8f8f7dd9 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:32.431355 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:32.435370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 425/848] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:32.441320 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b939b-0720-48e8-b59f-d5cace9ee3eb None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:32.464125 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:32.494829 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b326f-7987-4aac-9205-9d66ffbb682c None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:32.495155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b326f-7987-4aac-9205-9d66ffbb682c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:32.495432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b326f-7987-4aac-9205-9d66ffbb682c None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:32.497039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 426/849] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:55:32.505033 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bbddd-8d54-436c-8437-6a2e59a5102f None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:32.505170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bbddd-8d54-436c-8437-6a2e59a5102f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:32.505365 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4bbddd-8d54-436c-8437-6a2e59a5102f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:32.524790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-7d4bbddd-8d54-436c-8437-6a2e59a5102f None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:32.525965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-7d4bbddd-8d54-436c-8437-6a2e59a5102f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:32.558119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178b939b-0720-48e8-b59f-d5cace9ee3eb None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:32.597257 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1a347-fddd-49b7-bdca-48119cb708f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:32.598211 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 427/850] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 15:55:32.598544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1a347-fddd-49b7-bdca-48119cb708f8 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:32.599056 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1a347-fddd-49b7-bdca-48119cb708f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:32.599586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1a347-fddd-49b7-bdca-48119cb708f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:32.610052 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1a347-fddd-49b7-bdca-48119cb708f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:32.611960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1a347-fddd-49b7-bdca-48119cb708f8 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:32.623691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-9ef1a347-fddd-49b7-bdca-48119cb708f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:32.623691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:32.623691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:32.623691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:32.623691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:32.623691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:32.623691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:32.623691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:32.623691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:32.623691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:32.623691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:32.623691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:32.623691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:32.623691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:32.623691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:32.624580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:32.625478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:32.625478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:32.625478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:32.625478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:32.625478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:32.625478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:32.625478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:32.625478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:32.628046 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 424/851] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:32.634210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bbddd-8d54-436c-8437-6a2e59a5102f None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:32.735073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4bbddd-8d54-436c-8437-6a2e59a5102f None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:32.764294 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:32.771645 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01786743-30f8-4303-aa1f-fa8316f64b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:32.772517 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01786743-30f8-4303-aa1f-fa8316f64b2c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:32.772982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01786743-30f8-4303-aa1f-fa8316f64b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:32.773437 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01786743-30f8-4303-aa1f-fa8316f64b2c None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:32.774378 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01786743-30f8-4303-aa1f-fa8316f64b2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:32.774868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01786743-30f8-4303-aa1f-fa8316f64b2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:32.776549 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01786743-30f8-4303-aa1f-fa8316f64b2c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:32.789187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 428/852] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:32.795493 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1a347-fddd-49b7-bdca-48119cb708f8 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:32.875086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:32.887617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01786743-30f8-4303-aa1f-fa8316f64b2c None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:32.897062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1a347-fddd-49b7-bdca-48119cb708f8 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:32.935465 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ffc1d-0d1a-4fa6-afa7-2c498ab788e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:32.936140 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ffc1d-0d1a-4fa6-afa7-2c498ab788e1 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:32.936669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ffc1d-0d1a-4fa6-afa7-2c498ab788e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:32.936777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ffc1d-0d1a-4fa6-afa7-2c498ab788e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:32.937531 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ffc1d-0d1a-4fa6-afa7-2c498ab788e1 None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=de54687547a14ab89cc41a1fabaab400) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:32.937905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ffc1d-0d1a-4fa6-afa7-2c498ab788e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:32.939374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846ffc1d-0d1a-4fa6-afa7-2c498ab788e1 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:32.947019 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 425/853] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:55:32 2022] GET /identity/v3/role_assignments?user.id=de54687547a14ab89cc41a1fabaab400&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:33.004265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01786743-30f8-4303-aa1f-fa8316f64b2c None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:33.044036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c0b74-0ac9-4b1b-8a30-f4d54e7923f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:33.045137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c0b74-0ac9-4b1b-8a30-f4d54e7923f3 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:33.045619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c0b74-0ac9-4b1b-8a30-f4d54e7923f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:33.046095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425c0b74-0ac9-4b1b-8a30-f4d54e7923f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:33.058908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425c0b74-0ac9-4b1b-8a30-f4d54e7923f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-425c0b74-0ac9-4b1b-8a30-f4d54e7923f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:33.064846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:33.065831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:33.066684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:33.066684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:33.066684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:33.066684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:33.066684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:33.066684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:33.066684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:33.066684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:33.066684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:33.066684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:33.066684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:55:33.066684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:33.069568 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 429/854] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:55:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:33.075941 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ffc1d-0d1a-4fa6-afa7-2c498ab788e1 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:33.182151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ffc1d-0d1a-4fa6-afa7-2c498ab788e1 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:33.216853 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0895c487-2eff-473c-8109-b60371ce3d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:33.217463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895c487-2eff-473c-8109-b60371ce3d7b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:33.217735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895c487-2eff-473c-8109-b60371ce3d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:33.218067 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0895c487-2eff-473c-8109-b60371ce3d7b None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:33.218734 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895c487-2eff-473c-8109-b60371ce3d7b None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:33.219029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895c487-2eff-473c-8109-b60371ce3d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:33.220424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0895c487-2eff-473c-8109-b60371ce3d7b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:33.229453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 426/855] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:55:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:33.235152 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c0b74-0ac9-4b1b-8a30-f4d54e7923f3 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:33.289826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:33.345927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425c0b74-0ac9-4b1b-8a30-f4d54e7923f3 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:33.385210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e2c4f-108a-4595-9974-ad23b140bb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:33.385871 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e2c4f-108a-4595-9974-ad23b140bb56 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:33.386140 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e2c4f-108a-4595-9974-ad23b140bb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:33.386404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e2c4f-108a-4595-9974-ad23b140bb56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:33.398274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e2c4f-108a-4595-9974-ad23b140bb56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:33.399773 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e2c4f-108a-4595-9974-ad23b140bb56 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:33.405669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8372767-7cfd-4ba5-9312-3a836b25cbba None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:33.406174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8372767-7cfd-4ba5-9312-3a836b25cbba None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:33.406361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8372767-7cfd-4ba5-9312-3a836b25cbba None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:33.407974 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 427/856] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:55:33.413218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13316c-1387-46b4-a07f-63cbb06fa423 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:33.413590 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13316c-1387-46b4-a07f-63cbb06fa423 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:33.413863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed13316c-1387-46b4-a07f-63cbb06fa423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-3c5e2c4f-108a-4595-9974-ad23b140bb56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:33.414667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:33.415665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:33.416958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:33.416958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:33.416958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:33.416958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 430/857] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:33.421467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e2c4f-108a-4595-9974-ad23b140bb56 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:33.423927 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:33.435515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-ed13316c-1387-46b4-a07f-63cbb06fa423 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:33.437065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ed13316c-1387-46b4-a07f-63cbb06fa423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:33.509620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 428/858] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 15:55:33.510901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c411a-9127-488e-b56f-7e057061603a None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:33.511185 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c411a-9127-488e-b56f-7e057061603a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:33.511466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c411a-9127-488e-b56f-7e057061603a None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:33.512973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 429/859] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:33.521091 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8c2c1-ee74-43b3-aff0-4b6be35e3390 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:33.521488 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8c2c1-ee74-43b3-aff0-4b6be35e3390 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:33.521875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e8c2c1-ee74-43b3-aff0-4b6be35e3390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:33.541402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-86e8c2c1-ee74-43b3-aff0-4b6be35e3390 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:33.542744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-86e8c2c1-ee74-43b3-aff0-4b6be35e3390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:33.543423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e2c4f-108a-4595-9974-ad23b140bb56 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:33.545227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:33.580512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052305c-a167-487e-a986-b98b0926de4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:33.581473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052305c-a167-487e-a986-b98b0926de4a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:33.582011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052305c-a167-487e-a986-b98b0926de4a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:33.582494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052305c-a167-487e-a986-b98b0926de4a None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:33.583285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052305c-a167-487e-a986-b98b0926de4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:33.583757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052305c-a167-487e-a986-b98b0926de4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:33.585415 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052305c-a167-487e-a986-b98b0926de4a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:33.599276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 431/860] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:33.604546 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052305c-a167-487e-a986-b98b0926de4a None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:33.631582 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 430/861] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:33.712301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052305c-a167-487e-a986-b98b0926de4a None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:33.751314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6062ff3-cd16-4d34-b757-e2f0b315cca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:33.752324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6062ff3-cd16-4d34-b757-e2f0b315cca5 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:33.752784 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6062ff3-cd16-4d34-b757-e2f0b315cca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:33.754083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6062ff3-cd16-4d34-b757-e2f0b315cca5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:33.761592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6062ff3-cd16-4d34-b757-e2f0b315cca5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:33.763034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6062ff3-cd16-4d34-b757-e2f0b315cca5 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-d6062ff3-cd16-4d34-b757-e2f0b315cca5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:33.774770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:33.775750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:33.777351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:33.777351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:33.777351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:33.777351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:33.777351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:33.777351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:33.777351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:33.777351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:33.777351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 15:55:33.777351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:33.777351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 432/862] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Sep 8 15:55:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:33.781924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8c2c1-ee74-43b3-aff0-4b6be35e3390 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:33.796908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6062ff3-cd16-4d34-b757-e2f0b315cca5 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:33.824455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:33.849761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:33.888757 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e8c2c1-ee74-43b3-aff0-4b6be35e3390 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:33.904162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6062ff3-cd16-4d34-b757-e2f0b315cca5 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:33.929633 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f95ad-e86f-4c61-bf6f-f80d9c8548c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:33.930412 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f95ad-e86f-4c61-bf6f-f80d9c8548c4 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:33.930787 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f95ad-e86f-4c61-bf6f-f80d9c8548c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:33.931151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f95ad-e86f-4c61-bf6f-f80d9c8548c4 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:33.932050 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f95ad-e86f-4c61-bf6f-f80d9c8548c4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:33.932987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f95ad-e86f-4c61-bf6f-f80d9c8548c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:33.934855 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f95ad-e86f-4c61-bf6f-f80d9c8548c4 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:33.948972 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1745582-c147-43f4-8e9c-9b97a54c4d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:33.949829 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1745582-c147-43f4-8e9c-9b97a54c4d45 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:33.950242 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1745582-c147-43f4-8e9c-9b97a54c4d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:33.951122 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1745582-c147-43f4-8e9c-9b97a54c4d45 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:33.951881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1745582-c147-43f4-8e9c-9b97a54c4d45 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:33.953667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1745582-c147-43f4-8e9c-9b97a54c4d45 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:33.958758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 431/863] 10.222.0.27 () {56 vars in 1149 bytes} [Thu Sep 8 15:55:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:33.958758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f95ad-e86f-4c61-bf6f-f80d9c8548c4 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-a1745582-c147-43f4-8e9c-9b97a54c4d45 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:33.968173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:33.969144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:33.970032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:33.970032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:33.970032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:33.970032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:33.970032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:33.970032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:33.970032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:33.970032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:33.970032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:33.970032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:33.970032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:33.970032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:33.970032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 15:55:33.970032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:33.970032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 433/864] 10.222.0.27 () {56 vars in 1132 bytes} [Thu Sep 8 15:55:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:33.970881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1745582-c147-43f4-8e9c-9b97a54c4d45 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:34.057012 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:34.068804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f95ad-e86f-4c61-bf6f-f80d9c8548c4 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:34.073151 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1745582-c147-43f4-8e9c-9b97a54c4d45 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:34.093021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:34.105523 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593caede-4675-4705-a226-23dbb23c25a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:34.106345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593caede-4675-4705-a226-23dbb23c25a6 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:34.106671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593caede-4675-4705-a226-23dbb23c25a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:34.106990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593caede-4675-4705-a226-23dbb23c25a6 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:34.107742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593caede-4675-4705-a226-23dbb23c25a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:34.108045 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bfc82-7250-4211-bc3e-db9a7868c5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:34.108645 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bfc82-7250-4211-bc3e-db9a7868c5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:34.108915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bfc82-7250-4211-bc3e-db9a7868c5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:34.109099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593caede-4675-4705-a226-23dbb23c25a6 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:34.109195 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5bfc82-7250-4211-bc3e-db9a7868c5b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:34.124665 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 432/865] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Sep 8 15:55:33 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:55:34.125483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593caede-4675-4705-a226-23dbb23c25a6 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:34.191384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5bfc82-7250-4211-bc3e-db9a7868c5b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:34.193425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5bfc82-7250-4211-bc3e-db9a7868c5b8 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:34.272396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593caede-4675-4705-a226-23dbb23c25a6 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:34.300069 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:34.318639 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91a067-b911-4df7-b03d-62e52634ec45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:34.319311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91a067-b911-4df7-b03d-62e52634ec45 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:34.319622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91a067-b911-4df7-b03d-62e52634ec45 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:34.319895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef91a067-b911-4df7-b03d-62e52634ec45 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:34.320544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91a067-b911-4df7-b03d-62e52634ec45 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:34.320848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91a067-b911-4df7-b03d-62e52634ec45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:34.322363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef91a067-b911-4df7-b03d-62e52634ec45 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:34.322621 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 434/866] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:55:33 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:55:34.327402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 433/867] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Sep 8 15:55:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:34.328800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bfc82-7250-4211-bc3e-db9a7868c5b8 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:34.334234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91a067-b911-4df7-b03d-62e52634ec45 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:34.379972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:34.443239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5bfc82-7250-4211-bc3e-db9a7868c5b8 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:34.450362 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef91a067-b911-4df7-b03d-62e52634ec45 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:34.479718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b725e-7036-4c6d-84b4-8f02b3fe0b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:34.480670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b725e-7036-4c6d-84b4-8f02b3fe0b88 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:34.481111 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b725e-7036-4c6d-84b4-8f02b3fe0b88 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:34.481553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b725e-7036-4c6d-84b4-8f02b3fe0b88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:34.489779 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362042e-5e68-44cd-9ff8-be46735fa4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:34.490539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362042e-5e68-44cd-9ff8-be46735fa4ff None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:34.491253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362042e-5e68-44cd-9ff8-be46735fa4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:34.491674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362042e-5e68-44cd-9ff8-be46735fa4ff None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:34.492697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362042e-5e68-44cd-9ff8-be46735fa4ff None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:34.493100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362042e-5e68-44cd-9ff8-be46735fa4ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:34.493352 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b725e-7036-4c6d-84b4-8f02b3fe0b88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:34.494669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362042e-5e68-44cd-9ff8-be46735fa4ff None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:34.495075 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b725e-7036-4c6d-84b4-8f02b3fe0b88 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:34.500042 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 434/868] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Sep 8 15:55:34 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-038b725e-7036-4c6d-84b4-8f02b3fe0b88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:34.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:34.517919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:34.519021 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:34.519021 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:34.519021 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:34.520583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 435/869] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:34.528933 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362042e-5e68-44cd-9ff8-be46735fa4ff None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:34.642441 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362042e-5e68-44cd-9ff8-be46735fa4ff None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:34.650431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:34.681848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db539e9-e994-4cb1-8bcd-562558b59945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:34.682827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db539e9-e994-4cb1-8bcd-562558b59945 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:34.683327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db539e9-e994-4cb1-8bcd-562558b59945 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:34.683826 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db539e9-e994-4cb1-8bcd-562558b59945 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:34.684519 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db539e9-e994-4cb1-8bcd-562558b59945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:34.684830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db539e9-e994-4cb1-8bcd-562558b59945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:34.686215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db539e9-e994-4cb1-8bcd-562558b59945 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:34.699358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 435/870] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:34.705845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b725e-7036-4c6d-84b4-8f02b3fe0b88 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:34.750701 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:34.812331 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b725e-7036-4c6d-84b4-8f02b3fe0b88 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:34.847061 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d576bf-03f4-4d68-8ccd-8350223a8997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:34.847780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d576bf-03f4-4d68-8ccd-8350223a8997 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:34.848053 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d576bf-03f4-4d68-8ccd-8350223a8997 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:34.848361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d576bf-03f4-4d68-8ccd-8350223a8997 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:34.855255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d576bf-03f4-4d68-8ccd-8350223a8997 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:34.856751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d576bf-03f4-4d68-8ccd-8350223a8997 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:34.869413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-94d576bf-03f4-4d68-8ccd-8350223a8997 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:34.869413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:34.869413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:34.869413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:34.869413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:34.869413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:34.869413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:34.869413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:34.869413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:34.869413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:34.869413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:34.869413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:34.869413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:34.869413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:34.869413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:34.870364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:34.871254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:34.871254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:34.871254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:34.871254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:34.871254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:34.871254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:34.871254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:34.871254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:34.871254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 436/871] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:34.876658 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db539e9-e994-4cb1-8bcd-562558b59945 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:34.987600 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db539e9-e994-4cb1-8bcd-562558b59945 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:35.022935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9874f166-445f-4327-b77e-1a707725aedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:35.023599 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874f166-445f-4327-b77e-1a707725aedb None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:35.023928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874f166-445f-4327-b77e-1a707725aedb None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:35.024204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9874f166-445f-4327-b77e-1a707725aedb None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:35.024912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874f166-445f-4327-b77e-1a707725aedb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:35.025212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874f166-445f-4327-b77e-1a707725aedb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:35.026660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9874f166-445f-4327-b77e-1a707725aedb None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:35.039085 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 436/872] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:35.045131 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d576bf-03f4-4d68-8ccd-8350223a8997 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:35.149653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:35.156119 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d576bf-03f4-4d68-8ccd-8350223a8997 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:35.196112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb8dfd-5915-40df-953b-f6723b0e6809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:35.196731 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb8dfd-5915-40df-953b-f6723b0e6809 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:35.197020 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb8dfd-5915-40df-953b-f6723b0e6809 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:35.197262 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb8dfd-5915-40df-953b-f6723b0e6809 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:35.198119 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb8dfd-5915-40df-953b-f6723b0e6809 None admin] RBAC: Adding query filter params (role.id=b256252b45e541a491376a134f502183, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=b4324e4c04e343c98d9bb216c542fcba) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:35.198426 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb8dfd-5915-40df-953b-f6723b0e6809 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:35.199953 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb8dfd-5915-40df-953b-f6723b0e6809 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:35.204596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efa3e6-5445-4118-9986-ef0f737c65e3 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:35.204949 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efa3e6-5445-4118-9986-ef0f737c65e3 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:35.205176 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efa3e6-5445-4118-9986-ef0f737c65e3 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:35.206835 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 437/873] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:55:35.207304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 437/874] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:55:35 2022] GET /identity/v3/role_assignments?user.id=b4324e4c04e343c98d9bb216c542fcba&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=b256252b45e541a491376a134f502183 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:35.213354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5d803-4f95-4614-8d3b-43a36edcaac7 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:35.213712 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5d803-4f95-4614-8d3b-43a36edcaac7 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:35.214054 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5d803-4f95-4614-8d3b-43a36edcaac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:35.237691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-f0b5d803-4f95-4614-8d3b-43a36edcaac7 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:35.239034 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f0b5d803-4f95-4614-8d3b-43a36edcaac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:35.263286 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:35.316992 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 438/875] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:35 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:35.577733 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb8dfd-5915-40df-953b-f6723b0e6809 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:35.687878 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb8dfd-5915-40df-953b-f6723b0e6809 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:35.722651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b45638-8734-46f7-a04e-11ba40c65f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:35.723724 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b45638-8734-46f7-a04e-11ba40c65f4b None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:35.724229 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b45638-8734-46f7-a04e-11ba40c65f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:35.724695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b45638-8734-46f7-a04e-11ba40c65f4b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:35.725468 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b45638-8734-46f7-a04e-11ba40c65f4b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:35.727101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b45638-8734-46f7-a04e-11ba40c65f4b None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-e0b45638-8734-46f7-a04e-11ba40c65f4b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:35.736171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:35.737230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:35.738150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:35.738150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:35.738150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:35.738150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:35.738150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:35.738150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:35.738150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:35.738150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:35.738150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:35.738150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:35.738150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:35.738150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:35.738150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 15:55:35.738150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:35.740907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 438/876] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Sep 8 15:55:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:35.747017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5d803-4f95-4614-8d3b-43a36edcaac7 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:35.851876 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5d803-4f95-4614-8d3b-43a36edcaac7 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:35.887216 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8578be-2f52-46e3-8389-d79a04c7dd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:35.887865 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8578be-2f52-46e3-8389-d79a04c7dd98 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:35.888147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8578be-2f52-46e3-8389-d79a04c7dd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:35.888418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8578be-2f52-46e3-8389-d79a04c7dd98 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:35.889058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8578be-2f52-46e3-8389-d79a04c7dd98 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:35.889361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8578be-2f52-46e3-8389-d79a04c7dd98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:35.890743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8578be-2f52-46e3-8389-d79a04c7dd98 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:35.896112 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 439/877] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Sep 8 15:55:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:35.901667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b45638-8734-46f7-a04e-11ba40c65f4b None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:35.932037 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:36.005209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b45638-8734-46f7-a04e-11ba40c65f4b None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:36.043476 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e73f4d4-4320-407b-9ceb-6f9130871a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:36.043476 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73f4d4-4320-407b-9ceb-6f9130871a64 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:36.043476 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73f4d4-4320-407b-9ceb-6f9130871a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:36.043476 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e73f4d4-4320-407b-9ceb-6f9130871a64 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:36.043924 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73f4d4-4320-407b-9ceb-6f9130871a64 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:36.043924 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73f4d4-4320-407b-9ceb-6f9130871a64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:36.043924 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e73f4d4-4320-407b-9ceb-6f9130871a64 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:36.048941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 439/878] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Sep 8 15:55:35 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:36.433621 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b09cc3-06c0-41e6-aeb6-8c5990abaec8 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:36.434066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b09cc3-06c0-41e6-aeb6-8c5990abaec8 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:36.434182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b09cc3-06c0-41e6-aeb6-8c5990abaec8 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:36.435766 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 440/879] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:36.441267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e9e3f-b10d-406d-aefe-8a3fd3858cc8 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:36.441547 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e9e3f-b10d-406d-aefe-8a3fd3858cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:36.441815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e9e3f-b10d-406d-aefe-8a3fd3858cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:36.460113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-679e9e3f-b10d-406d-aefe-8a3fd3858cc8 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:36.461635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-679e9e3f-b10d-406d-aefe-8a3fd3858cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:36.528822 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 440/880] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:36 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:36.764230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc45d6a-815e-4ea0-a55e-78297a61edeb None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:36.764750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc45d6a-815e-4ea0-a55e-78297a61edeb None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:36.764750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc45d6a-815e-4ea0-a55e-78297a61edeb None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:36.766351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 441/881] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:36.771811 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22edc1d-074b-4df2-9fb3-49d6974a82cc None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:36.772093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22edc1d-074b-4df2-9fb3-49d6974a82cc None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:36.772372 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22edc1d-074b-4df2-9fb3-49d6974a82cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:36.790486 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-a22edc1d-074b-4df2-9fb3-49d6974a82cc None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:36.791703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a22edc1d-074b-4df2-9fb3-49d6974a82cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:36.804543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc45d6a-815e-4ea0-a55e-78297a61edeb None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:36.808059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:36.831817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:36.867949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 441/882] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:36 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:36.918610 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc45d6a-815e-4ea0-a55e-78297a61edeb None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:36.955555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c7a6f-2323-4ef8-a833-d02e9621f581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:36.956208 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c7a6f-2323-4ef8-a833-d02e9621f581 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:36.956487 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c7a6f-2323-4ef8-a833-d02e9621f581 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:36.956759 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c7a6f-2323-4ef8-a833-d02e9621f581 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:36.957476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c7a6f-2323-4ef8-a833-d02e9621f581 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:36.958851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c7a6f-2323-4ef8-a833-d02e9621f581 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-5c6c7a6f-2323-4ef8-a833-d02e9621f581 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:36.968152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:36.969160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:36.970663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:36.970663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:36.970663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:36.970663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:36.970663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:36.970663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:36.970663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:36.970663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:36.970663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:36.970663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:36.970663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:36.970663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:36.970663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 15:55:36.970663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:36.970663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 442/883] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:55:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:36.975258 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22edc1d-074b-4df2-9fb3-49d6974a82cc None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:37.076688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22edc1d-074b-4df2-9fb3-49d6974a82cc None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:37.111179 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbf10d-df8d-44e5-bf67-c1a3d80d2eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:37.111859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbf10d-df8d-44e5-bf67-c1a3d80d2eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:37.112157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbf10d-df8d-44e5-bf67-c1a3d80d2eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:37.112435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbf10d-df8d-44e5-bf67-c1a3d80d2eb0 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:37.113067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbf10d-df8d-44e5-bf67-c1a3d80d2eb0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:37.113362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbf10d-df8d-44e5-bf67-c1a3d80d2eb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:37.114738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbf10d-df8d-44e5-bf67-c1a3d80d2eb0 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:37.119747 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 442/884] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:55:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:37.128496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c7a6f-2323-4ef8-a833-d02e9621f581 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:37.141360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbf10d-df8d-44e5-bf67-c1a3d80d2eb0 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:37.162863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:37.241690 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:37.261605 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c7a6f-2323-4ef8-a833-d02e9621f581 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:37.265955 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbf10d-df8d-44e5-bf67-c1a3d80d2eb0 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:37.300795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11b3c9-fba6-4558-a8f5-d468ee76c737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:37.301740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7bd70-32fb-42db-bcdb-f1cf2c28d7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:37.302442 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7bd70-32fb-42db-bcdb-f1cf2c28d7bc None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:37.302678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11b3c9-fba6-4558-a8f5-d468ee76c737 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:37.302799 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7bd70-32fb-42db-bcdb-f1cf2c28d7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:37.303122 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7bd70-32fb-42db-bcdb-f1cf2c28d7bc None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:37.303630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11b3c9-fba6-4558-a8f5-d468ee76c737 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:37.303927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7bd70-32fb-42db-bcdb-f1cf2c28d7bc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:37.304260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11b3c9-fba6-4558-a8f5-d468ee76c737 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:37.305106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11b3c9-fba6-4558-a8f5-d468ee76c737 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:37.305496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7bd70-32fb-42db-bcdb-f1cf2c28d7bc None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:37.305777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11b3c9-fba6-4558-a8f5-d468ee76c737 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:37.307371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11b3c9-fba6-4558-a8f5-d468ee76c737 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:37.318509 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 443/885] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:55:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:37.319874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11b3c9-fba6-4558-a8f5-d468ee76c737 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:37.323576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 443/886] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Sep 8 15:55:37 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 15:55:37.441048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11b3c9-fba6-4558-a8f5-d468ee76c737 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:37.490941 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2128b3f1-b97a-45e3-acb5-429ef0ebe5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:37.491924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128b3f1-b97a-45e3-acb5-429ef0ebe5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:37.492377 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128b3f1-b97a-45e3-acb5-429ef0ebe5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:37.492929 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2128b3f1-b97a-45e3-acb5-429ef0ebe5a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:37.493817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128b3f1-b97a-45e3-acb5-429ef0ebe5a8 None admin] RBAC: Adding query filter params (interface=public, service_id=bac24a6c423c4df59fc87f13afb8efb1, region_id=RegionOne) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:37.494312 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128b3f1-b97a-45e3-acb5-429ef0ebe5a8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:37.495934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2128b3f1-b97a-45e3-acb5-429ef0ebe5a8 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:37.502994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 444/887] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Sep 8 15:55:37 2022] GET /identity/v3/endpoints?service_id=bac24a6c423c4df59fc87f13afb8efb1&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:37.510374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7bd70-32fb-42db-bcdb-f1cf2c28d7bc None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:37.575750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bda569-637f-4748-bfc7-e84b242af3e8 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:37.576671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bda569-637f-4748-bfc7-e84b242af3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:37.577069 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bda569-637f-4748-bfc7-e84b242af3e8 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:37.578854 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 445/888] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:37.584065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c10ec-84fe-46d4-a4a8-a7c3b9467d72 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:37.584342 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c10ec-84fe-46d4-a4a8-a7c3b9467d72 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:37.584608 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c10ec-84fe-46d4-a4a8-a7c3b9467d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:37.597320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:37.608685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-852c10ec-84fe-46d4-a4a8-a7c3b9467d72 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:37.616097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-852c10ec-84fe-46d4-a4a8-a7c3b9467d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:37.618600 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:37.637673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7bd70-32fb-42db-bcdb-f1cf2c28d7bc None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:37.680926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46c20b-4e96-4c18-89af-5a0bc5047a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:37.682301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46c20b-4e96-4c18-89af-5a0bc5047a3d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:37.682808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46c20b-4e96-4c18-89af-5a0bc5047a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:37.683423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d46c20b-4e96-4c18-89af-5a0bc5047a3d None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:37.684604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46c20b-4e96-4c18-89af-5a0bc5047a3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:37.686661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d46c20b-4e96-4c18-89af-5a0bc5047a3d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:37.694724 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 444/890] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Sep 8 15:55:37 2022] GET /identity/v3/services => generated 1268 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 15:55:37.703559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 446/890] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 15:55:37.959737 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46c20b-4e96-4c18-89af-5a0bc5047a3d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:38.063095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d46c20b-4e96-4c18-89af-5a0bc5047a3d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:38.101428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a3f2ff-67ab-42b0-86d8-c820efecdb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:38.102080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3f2ff-67ab-42b0-86d8-c820efecdb6e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:38.102343 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3f2ff-67ab-42b0-86d8-c820efecdb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:38.102622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a3f2ff-67ab-42b0-86d8-c820efecdb6e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:38.111164 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:38.117085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a3f2ff-67ab-42b0-86d8-c820efecdb6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-38a3f2ff-67ab-42b0-86d8-c820efecdb6e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:38.122204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:38.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:38.124767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:38.124767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:38.124767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:38.124767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:38.124767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:38.124767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:38.124767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:38.124767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:38.124767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:38.124767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:38.124767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 15:55:38.124767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:38.124767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 445/891] 10.222.0.27 () {56 vars in 1120 bytes} [Thu Sep 8 15:55:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:38.132645 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c10ec-84fe-46d4-a4a8-a7c3b9467d72 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:38.214330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55634b06-a131-4468-b01b-acb5f6116d38 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:38.214523 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55634b06-a131-4468-b01b-acb5f6116d38 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:38.214778 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55634b06-a131-4468-b01b-acb5f6116d38 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:38.216405 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 446/892] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:38.221406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef29c68-85ed-4f69-b89f-bb1a88f7e5c3 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:38.221681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef29c68-85ed-4f69-b89f-bb1a88f7e5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:38.221943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef29c68-85ed-4f69-b89f-bb1a88f7e5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:38.240154 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c10ec-84fe-46d4-a4a8-a7c3b9467d72 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:38.242135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-3ef29c68-85ed-4f69-b89f-bb1a88f7e5c3 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:38.243971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3ef29c68-85ed-4f69-b89f-bb1a88f7e5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:38.277560 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a7ab0-9a4e-48ae-a477-1825c3d464e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:38.278234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a7ab0-9a4e-48ae-a477-1825c3d464e1 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:38.278664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a7ab0-9a4e-48ae-a477-1825c3d464e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:38.278966 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a7ab0-9a4e-48ae-a477-1825c3d464e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:38.279759 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a7ab0-9a4e-48ae-a477-1825c3d464e1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:38.280121 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a7ab0-9a4e-48ae-a477-1825c3d464e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:38.281546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a7ab0-9a4e-48ae-a477-1825c3d464e1 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:38.286845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:38.290705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 447/893] 10.222.0.27 () {56 vars in 1115 bytes} [Thu Sep 8 15:55:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:38.296716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a7ab0-9a4e-48ae-a477-1825c3d464e1 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:38.334984 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 447/894] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:38 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 15:55:38.405358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a7ab0-9a4e-48ae-a477-1825c3d464e1 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:38.440961 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5bfdf-8784-43fa-addc-2c81cd3581b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:38.441986 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5bfdf-8784-43fa-addc-2c81cd3581b5 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:38.442479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5bfdf-8784-43fa-addc-2c81cd3581b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:38.442955 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5bfdf-8784-43fa-addc-2c81cd3581b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:38.454418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5bfdf-8784-43fa-addc-2c81cd3581b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:38.456194 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5bfdf-8784-43fa-addc-2c81cd3581b5 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-5fc5bfdf-8784-43fa-addc-2c81cd3581b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:38.471761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:38.472770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:38.473705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:38.473705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:38.473705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:38.475919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 448/895] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:38.481965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef29c68-85ed-4f69-b89f-bb1a88f7e5c3 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:38.586210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef29c68-85ed-4f69-b89f-bb1a88f7e5c3 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:38.598698 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5bfdf-8784-43fa-addc-2c81cd3581b5 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:38.622067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:38.631107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6adbc3-4a73-4543-a52e-5b5c39dcb915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:38.632014 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6adbc3-4a73-4543-a52e-5b5c39dcb915 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:38.632484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6adbc3-4a73-4543-a52e-5b5c39dcb915 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:38.633052 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6adbc3-4a73-4543-a52e-5b5c39dcb915 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:38.633978 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6adbc3-4a73-4543-a52e-5b5c39dcb915 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:38.634498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6adbc3-4a73-4543-a52e-5b5c39dcb915 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:38.636137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6adbc3-4a73-4543-a52e-5b5c39dcb915 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:38.648474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 448/896] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:38.654100 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6adbc3-4a73-4543-a52e-5b5c39dcb915 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:38.655265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:38.708175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5bfdf-8784-43fa-addc-2c81cd3581b5 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:38.753276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdb87c-b5f6-4ba1-a578-94dc6dc19afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:38.753793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdb87c-b5f6-4ba1-a578-94dc6dc19afa None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:38.754121 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdb87c-b5f6-4ba1-a578-94dc6dc19afa None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:38.754584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbdb87c-b5f6-4ba1-a578-94dc6dc19afa None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:38.755682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbdb87c-b5f6-4ba1-a578-94dc6dc19afa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:38.757304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbdb87c-b5f6-4ba1-a578-94dc6dc19afa None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:38.764362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6adbc3-4a73-4543-a52e-5b5c39dcb915 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:38.776035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 449/897] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Sep 8 15:55:38 2022] POST /identity/v3/services => generated 241 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 15:55:38.820663 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56411085-c088-4be8-bef5-7fd3b24f9912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:38.821782 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56411085-c088-4be8-bef5-7fd3b24f9912 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:38.822242 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56411085-c088-4be8-bef5-7fd3b24f9912 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:38.822794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56411085-c088-4be8-bef5-7fd3b24f9912 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:38.832670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56411085-c088-4be8-bef5-7fd3b24f9912 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:38.834551 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56411085-c088-4be8-bef5-7fd3b24f9912 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-56411085-c088-4be8-bef5-7fd3b24f9912 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:38.848894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:38.849981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:38.850952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:38.850952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:38.850952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:38.850952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:38.850952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:38.850952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:38.850952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:38.850952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:38.850952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 15:55:38.850952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:38.853471 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 449/898] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Sep 8 15:55:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:38.859753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdb87c-b5f6-4ba1-a578-94dc6dc19afa None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:38.975543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbdb87c-b5f6-4ba1-a578-94dc6dc19afa None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:38.989738 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:39.015823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40f90a-4756-4acd-8714-c79b699678b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:39.016366 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40f90a-4756-4acd-8714-c79b699678b0 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:39.016645 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40f90a-4756-4acd-8714-c79b699678b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:39.016897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40f90a-4756-4acd-8714-c79b699678b0 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:39.017682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40f90a-4756-4acd-8714-c79b699678b0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:39.017981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40f90a-4756-4acd-8714-c79b699678b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:39.019411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40f90a-4756-4acd-8714-c79b699678b0 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:39.037574 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 450/899] 10.222.0.27 () {56 vars in 1149 bytes} [Thu Sep 8 15:55:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:39.043639 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56411085-c088-4be8-bef5-7fd3b24f9912 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:39.092354 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:39.148898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56411085-c088-4be8-bef5-7fd3b24f9912 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:39.183756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a552f8-fad3-4de9-a478-6904e5b70f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:39.184379 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a552f8-fad3-4de9-a478-6904e5b70f86 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:39.184651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a552f8-fad3-4de9-a478-6904e5b70f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:39.184930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a552f8-fad3-4de9-a478-6904e5b70f86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:39.254412 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a552f8-fad3-4de9-a478-6904e5b70f86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:39.255924 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a552f8-fad3-4de9-a478-6904e5b70f86 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:39.308282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:39.361488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 450/900] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:55:39 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:55:39.367175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40f90a-4756-4acd-8714-c79b699678b0 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:39.471012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40f90a-4756-4acd-8714-c79b699678b0 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:39.506510 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7e2b9-b33c-40c6-a390-c8f8ef409b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:39.507197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7e2b9-b33c-40c6-a390-c8f8ef409b30 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:39.507496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7e2b9-b33c-40c6-a390-c8f8ef409b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:39.507770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7e2b9-b33c-40c6-a390-c8f8ef409b30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:39.520647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7e2b9-b33c-40c6-a390-c8f8ef409b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:39.522145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7e2b9-b33c-40c6-a390-c8f8ef409b30 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-5db7e2b9-b33c-40c6-a390-c8f8ef409b30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:39.540399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:39.541429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:39.542336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:39.542336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:39.542336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:39.542336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 451/901] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:39.547339 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a552f8-fad3-4de9-a478-6904e5b70f86 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:39.633284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b998c7-4826-4b64-95cc-d3a2e196c228 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:39.633504 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b998c7-4826-4b64-95cc-d3a2e196c228 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:39.633741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b998c7-4826-4b64-95cc-d3a2e196c228 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:39.636436 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 452/902] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:39.640469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea824a66-2dde-4161-b2c8-a6dec0afc065 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:39.640753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea824a66-2dde-4161-b2c8-a6dec0afc065 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:39.641029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea824a66-2dde-4161-b2c8-a6dec0afc065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:39.652653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a552f8-fad3-4de9-a478-6904e5b70f86 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:39.659767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-ea824a66-2dde-4161-b2c8-a6dec0afc065 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:39.660972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ea824a66-2dde-4161-b2c8-a6dec0afc065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:39.678252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:39.689136 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8d4aa-58e1-4fff-b8db-dd574d3067fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:39.689846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8d4aa-58e1-4fff-b8db-dd574d3067fb None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:39.690124 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8d4aa-58e1-4fff-b8db-dd574d3067fb None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:39.690407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8d4aa-58e1-4fff-b8db-dd574d3067fb None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:39.691043 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8d4aa-58e1-4fff-b8db-dd574d3067fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:39.691345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8d4aa-58e1-4fff-b8db-dd574d3067fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:39.692761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8d4aa-58e1-4fff-b8db-dd574d3067fb None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:39.705095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 451/903] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:39.710691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8d4aa-58e1-4fff-b8db-dd574d3067fb None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:39.714848 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:39.743578 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 453/904] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 15:55:39.824843 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8d4aa-58e1-4fff-b8db-dd574d3067fb None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:39.859474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead5e2f-6d59-46e6-ba9d-825d9f9e0dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:39.860132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead5e2f-6d59-46e6-ba9d-825d9f9e0dce None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:39.860404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead5e2f-6d59-46e6-ba9d-825d9f9e0dce None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:39.860669 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead5e2f-6d59-46e6-ba9d-825d9f9e0dce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:39.867679 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead5e2f-6d59-46e6-ba9d-825d9f9e0dce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:39.869133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead5e2f-6d59-46e6-ba9d-825d9f9e0dce None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:39.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-9ead5e2f-6d59-46e6-ba9d-825d9f9e0dce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:39.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:39.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:39.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:39.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:39.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:39.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:39.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:39.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:39.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:39.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:39.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:39.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:39.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:39.880424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:39.881323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:39.882210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:39.882210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:39.882210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:39.882210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:39.882210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:39.882210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:39.882210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:39.882210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:39.882210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 452/905] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:39.887692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea824a66-2dde-4161-b2c8-a6dec0afc065 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:39.894335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d69c41-d60d-4bb1-9a94-3d45adfae3f0 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:39.894606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d69c41-d60d-4bb1-9a94-3d45adfae3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:39.894875 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d69c41-d60d-4bb1-9a94-3d45adfae3f0 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:39.896461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 453/906] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:55:39.903257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71031d2a-864c-4217-809c-e0290b6e4b5d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:39.903648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71031d2a-864c-4217-809c-e0290b6e4b5d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:39.904004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71031d2a-864c-4217-809c-e0290b6e4b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:39.924230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-71031d2a-864c-4217-809c-e0290b6e4b5d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:39.925488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-71031d2a-864c-4217-809c-e0290b6e4b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:39.987665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:40.003856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 454/907] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 15:55:40.005711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea824a66-2dde-4161-b2c8-a6dec0afc065 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:40.020849 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71031d2a-864c-4217-809c-e0290b6e4b5d None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:40.041283 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa182fe-d2ba-4a6d-acdb-82b546feb733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:40.042142 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa182fe-d2ba-4a6d-acdb-82b546feb733 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:40.042616 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa182fe-d2ba-4a6d-acdb-82b546feb733 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:40.043053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa182fe-d2ba-4a6d-acdb-82b546feb733 None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:40.043964 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa182fe-d2ba-4a6d-acdb-82b546feb733 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:40.044444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa182fe-d2ba-4a6d-acdb-82b546feb733 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:40.045983 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa182fe-d2ba-4a6d-acdb-82b546feb733 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:40.058457 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 454/908] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:40.064592 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa182fe-d2ba-4a6d-acdb-82b546feb733 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:40.084935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:40.129879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71031d2a-864c-4217-809c-e0290b6e4b5d None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:40.177673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda8b5d-f9b6-4331-98aa-b0dce732cdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:40.178289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda8b5d-f9b6-4331-98aa-b0dce732cdb7 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:40.178546 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda8b5d-f9b6-4331-98aa-b0dce732cdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:40.178750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda8b5d-f9b6-4331-98aa-b0dce732cdb7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:40.182121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda8b5d-f9b6-4331-98aa-b0dce732cdb7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:40.182121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eda8b5d-f9b6-4331-98aa-b0dce732cdb7 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-4eda8b5d-f9b6-4331-98aa-b0dce732cdb7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:40.192074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:40.193029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:40.193928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:40.193928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:40.193928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:40.193928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:40.193928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:40.193928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:40.193928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:40.193928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:40.193928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:40.193928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:40.193928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:40.193928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:40.193928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 15:55:40.193928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:40.193928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 455/909] 10.222.0.27 () {56 vars in 1132 bytes} [Thu Sep 8 15:55:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:40.195362 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa182fe-d2ba-4a6d-acdb-82b546feb733 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:40.199324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda8b5d-f9b6-4331-98aa-b0dce732cdb7 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:40.230914 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e26a73-c443-4770-9010-d2c61f7abe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:40.231561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e26a73-c443-4770-9010-d2c61f7abe44 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:40.231884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e26a73-c443-4770-9010-d2c61f7abe44 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:40.232151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e26a73-c443-4770-9010-d2c61f7abe44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:40.232930 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e26a73-c443-4770-9010-d2c61f7abe44 None admin] RBAC: Adding query filter params (role.id=f752d851596f48a3bb97ad5d8f09d763, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=b4324e4c04e343c98d9bb216c542fcba) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:40.233232 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e26a73-c443-4770-9010-d2c61f7abe44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:40.235209 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e26a73-c443-4770-9010-d2c61f7abe44 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:40.241674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 455/910] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:55:40 2022] GET /identity/v3/role_assignments?user.id=b4324e4c04e343c98d9bb216c542fcba&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=f752d851596f48a3bb97ad5d8f09d763 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:40.279744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e26a73-c443-4770-9010-d2c61f7abe44 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:40.285513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:40.335351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda8b5d-f9b6-4331-98aa-b0dce732cdb7 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:40.373150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2c13a-1bee-4e64-a645-e3b00eb7dd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:40.373927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2c13a-1bee-4e64-a645-e3b00eb7dd62 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:40.374206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2c13a-1bee-4e64-a645-e3b00eb7dd62 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:40.374465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2c13a-1bee-4e64-a645-e3b00eb7dd62 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:40.375131 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad2c13a-1bee-4e64-a645-e3b00eb7dd62 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:40.375423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad2c13a-1bee-4e64-a645-e3b00eb7dd62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:40.377176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad2c13a-1bee-4e64-a645-e3b00eb7dd62 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:40.383037 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 456/911] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Sep 8 15:55:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:40.386795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:40.392257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2c13a-1bee-4e64-a645-e3b00eb7dd62 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:40.415014 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e26a73-c443-4770-9010-d2c61f7abe44 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:40.457584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7909b6b-ab43-45c4-8b11-45f1df1fc2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:40.458234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7909b6b-ab43-45c4-8b11-45f1df1fc2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:40.458583 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7909b6b-ab43-45c4-8b11-45f1df1fc2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:40.458935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7909b6b-ab43-45c4-8b11-45f1df1fc2b6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:40.459663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7909b6b-ab43-45c4-8b11-45f1df1fc2b6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:40.461099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7909b6b-ab43-45c4-8b11-45f1df1fc2b6 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-c7909b6b-ab43-45c4-8b11-45f1df1fc2b6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:40.471845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:40.473058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:40.473898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:40.473898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:40.473898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:40.473898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:40.473898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:40.473898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:40.473898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:40.473898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:40.473898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:40.473898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:40.473898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:40.473898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:40.473898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 15:55:40.473898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:40.481353 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 456/912] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:55:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:40.482819 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7909b6b-ab43-45c4-8b11-45f1df1fc2b6 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:40.527431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2c13a-1bee-4e64-a645-e3b00eb7dd62 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:40.557725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:40.582847 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182f28c-7376-42d6-ab71-8362a47be326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:40.583244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182f28c-7376-42d6-ab71-8362a47be326 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:40.587073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182f28c-7376-42d6-ab71-8362a47be326 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:40.587073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8182f28c-7376-42d6-ab71-8362a47be326 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:40.587647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182f28c-7376-42d6-ab71-8362a47be326 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:40.587951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182f28c-7376-42d6-ab71-8362a47be326 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:40.589358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8182f28c-7376-42d6-ab71-8362a47be326 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:40.599553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 457/913] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Sep 8 15:55:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:40.601859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182f28c-7376-42d6-ab71-8362a47be326 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:40.608694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7909b6b-ab43-45c4-8b11-45f1df1fc2b6 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:40.656683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b78a2-4649-439c-bcc9-89c243b51f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:40.657588 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b78a2-4649-439c-bcc9-89c243b51f6f None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:40.658485 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b78a2-4649-439c-bcc9-89c243b51f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:40.658907 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552b78a2-4649-439c-bcc9-89c243b51f6f None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:40.659742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b78a2-4649-439c-bcc9-89c243b51f6f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:40.660546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b78a2-4649-439c-bcc9-89c243b51f6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:40.662376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552b78a2-4649-439c-bcc9-89c243b51f6f None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:40.669594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 457/914] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:55:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:40.675469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b78a2-4649-439c-bcc9-89c243b51f6f None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:40.681568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:40.737217 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8182f28c-7376-42d6-ab71-8362a47be326 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:40.782240 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f75097-6fe9-44c0-acfc-d8ec7bc945c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:40.783471 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f75097-6fe9-44c0-acfc-d8ec7bc945c0 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:40.783844 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f75097-6fe9-44c0-acfc-d8ec7bc945c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:40.784472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f75097-6fe9-44c0-acfc-d8ec7bc945c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:40.785276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f75097-6fe9-44c0-acfc-d8ec7bc945c0 None admin] RBAC: Adding query filter params (interface=public, service_id=47247ed04f0b4202bb528813eee4c834, region_id=RegionOne) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:40.785917 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f75097-6fe9-44c0-acfc-d8ec7bc945c0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:40.788051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f75097-6fe9-44c0-acfc-d8ec7bc945c0 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:40.789792 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552b78a2-4649-439c-bcc9-89c243b51f6f None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:40.794240 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 458/915] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Sep 8 15:55:40 2022] GET /identity/v3/endpoints?service_id=47247ed04f0b4202bb528813eee4c834&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:40.800312 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f75097-6fe9-44c0-acfc-d8ec7bc945c0 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:40.832888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc894c8f-201c-4d0f-8330-df67fca38075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:40.833753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc894c8f-201c-4d0f-8330-df67fca38075 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:40.834196 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc894c8f-201c-4d0f-8330-df67fca38075 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:40.834646 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc894c8f-201c-4d0f-8330-df67fca38075 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:40.835601 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc894c8f-201c-4d0f-8330-df67fca38075 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:40.836136 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc894c8f-201c-4d0f-8330-df67fca38075 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:40.838047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc894c8f-201c-4d0f-8330-df67fca38075 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:40.848540 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 458/916] 10.222.0.27 () {56 vars in 1114 bytes} [Thu Sep 8 15:55:40 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:40.849892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc894c8f-201c-4d0f-8330-df67fca38075 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:40.859299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:40.922195 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f75097-6fe9-44c0-acfc-d8ec7bc945c0 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:40.944293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:40.956866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e681f-0d13-48d0-832f-e31de9baa34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:40.957637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e681f-0d13-48d0-832f-e31de9baa34a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:40.958020 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e681f-0d13-48d0-832f-e31de9baa34a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:40.958386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e681f-0d13-48d0-832f-e31de9baa34a None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:40.959229 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e681f-0d13-48d0-832f-e31de9baa34a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:40.960848 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e681f-0d13-48d0-832f-e31de9baa34a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:40.975988 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 459/917] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Sep 8 15:55:40 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 15:55:40.989214 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc894c8f-201c-4d0f-8330-df67fca38075 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:41.030122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17086c54-aee9-43cf-baf9-b6666bf25c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:41.031048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17086c54-aee9-43cf-baf9-b6666bf25c6d None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:41.031503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17086c54-aee9-43cf-baf9-b6666bf25c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:41.031963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17086c54-aee9-43cf-baf9-b6666bf25c6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:41.033016 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17086c54-aee9-43cf-baf9-b6666bf25c6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:41.034635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17086c54-aee9-43cf-baf9-b6666bf25c6d None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:41.068047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 459/918] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 15:55:40 2022] POST /identity/v3/endpoints => generated 346 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 15:55:41.101130 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a5883-62db-4e9c-b476-2b99eee20359 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:41.101576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a5883-62db-4e9c-b476-2b99eee20359 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:41.101947 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a5883-62db-4e9c-b476-2b99eee20359 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:41.103835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 460/919] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:41.110317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50f57c-1ea0-40f9-bc5b-2042d18d9dc9 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:41.110605 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50f57c-1ea0-40f9-bc5b-2042d18d9dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:41.111091 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50f57c-1ea0-40f9-bc5b-2042d18d9dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:41.132749 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-4d50f57c-1ea0-40f9-bc5b-2042d18d9dc9 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:41.134101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4d50f57c-1ea0-40f9-bc5b-2042d18d9dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:41.229174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 460/920] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:41.498473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a5883-62db-4e9c-b476-2b99eee20359 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:41.604854 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a5883-62db-4e9c-b476-2b99eee20359 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:41.619506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:41.640933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ab172-73f6-4ee5-bc1a-d83244e53944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:41.641568 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ab172-73f6-4ee5-bc1a-d83244e53944 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:41.641843 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ab172-73f6-4ee5-bc1a-d83244e53944 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:41.642137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ab172-73f6-4ee5-bc1a-d83244e53944 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:41.642749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ab172-73f6-4ee5-bc1a-d83244e53944 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:41.644169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ab172-73f6-4ee5-bc1a-d83244e53944 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-835ab172-73f6-4ee5-bc1a-d83244e53944 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:41.652999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:41.653991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:41.655349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:41.655349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:41.655349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:41.655349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:41.655349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:41.655349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:41.655349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:41.655349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:41.655349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:41.655349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:41.655349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:41.655349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:41.655349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 15:55:41.655349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:41.655349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 461/921] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Sep 8 15:55:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:41.660427 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50f57c-1ea0-40f9-bc5b-2042d18d9dc9 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:41.711973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:41.762866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50f57c-1ea0-40f9-bc5b-2042d18d9dc9 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:41.798061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975355d-3e85-421d-884f-78f88df9b419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:41.798686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975355d-3e85-421d-884f-78f88df9b419 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:41.798959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975355d-3e85-421d-884f-78f88df9b419 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:41.799224 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975355d-3e85-421d-884f-78f88df9b419 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:41.799887 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975355d-3e85-421d-884f-78f88df9b419 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:41.800181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975355d-3e85-421d-884f-78f88df9b419 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:41.801558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a975355d-3e85-421d-884f-78f88df9b419 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:41.807018 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 461/922] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Sep 8 15:55:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:41.812767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ab172-73f6-4ee5-bc1a-d83244e53944 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:41.916995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ab172-73f6-4ee5-bc1a-d83244e53944 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:41.955730 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8985f47-cbdc-4b1d-b610-ec4813fa2ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:41.956834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8985f47-cbdc-4b1d-b610-ec4813fa2ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:41.957321 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8985f47-cbdc-4b1d-b610-ec4813fa2ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:41.957791 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8985f47-cbdc-4b1d-b610-ec4813fa2ee9 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:41.958630 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8985f47-cbdc-4b1d-b610-ec4813fa2ee9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:41.959130 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8985f47-cbdc-4b1d-b610-ec4813fa2ee9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:41.960761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8985f47-cbdc-4b1d-b610-ec4813fa2ee9 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:41.966914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 462/923] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Sep 8 15:55:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:41.975477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975355d-3e85-421d-884f-78f88df9b419 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:42.077007 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975355d-3e85-421d-884f-78f88df9b419 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:42.092070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:42.112704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6fa8a-8efd-4340-b50f-64f5e66e4907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:42.113313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6fa8a-8efd-4340-b50f-64f5e66e4907 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:42.113583 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6fa8a-8efd-4340-b50f-64f5e66e4907 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:42.113847 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6fa8a-8efd-4340-b50f-64f5e66e4907 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:42.114538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6fa8a-8efd-4340-b50f-64f5e66e4907 None admin] RBAC: Adding query filter params (interface=public, service_id=5d6784bcb33a48eaafdbdd21c3435368, region_id=RegionOne) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:42.114837 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6fa8a-8efd-4340-b50f-64f5e66e4907 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:42.116227 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6fa8a-8efd-4340-b50f-64f5e66e4907 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:42.121753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 462/924] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Sep 8 15:55:41 2022] GET /identity/v3/endpoints?service_id=5d6784bcb33a48eaafdbdd21c3435368&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:42.127539 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8985f47-cbdc-4b1d-b610-ec4813fa2ee9 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:42.174788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:42.232543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8985f47-cbdc-4b1d-b610-ec4813fa2ee9 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:42.272007 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6941e9-bafa-43dd-8c42-1e24248416d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:42.272623 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6941e9-bafa-43dd-8c42-1e24248416d6 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:42.272895 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6941e9-bafa-43dd-8c42-1e24248416d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:42.273163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6941e9-bafa-43dd-8c42-1e24248416d6 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:42.273770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6941e9-bafa-43dd-8c42-1e24248416d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:42.275133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6941e9-bafa-43dd-8c42-1e24248416d6 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:42.281557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 463/925] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Sep 8 15:55:42 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 15:55:42.424978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3273da-478c-4aa6-9b69-9ae937ba4d5d None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:42.425160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3273da-478c-4aa6-9b69-9ae937ba4d5d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:42.425432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3273da-478c-4aa6-9b69-9ae937ba4d5d None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:42.427045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 463/926] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:42.432904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb9c28-50a0-4b3f-be17-6b5fe42f59db None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:42.433206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb9c28-50a0-4b3f-be17-6b5fe42f59db None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:42.433450 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb9c28-50a0-4b3f-be17-6b5fe42f59db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:42.453163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-3feb9c28-50a0-4b3f-be17-6b5fe42f59db None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:42.454380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3feb9c28-50a0-4b3f-be17-6b5fe42f59db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:42.525076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 464/927] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:42.789821 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3273da-478c-4aa6-9b69-9ae937ba4d5d None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:42.897199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3273da-478c-4aa6-9b69-9ae937ba4d5d None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:42.932177 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee8e9b-77eb-4ae4-b71a-8adac1ae7d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:42.933209 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee8e9b-77eb-4ae4-b71a-8adac1ae7d5b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:42.933678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee8e9b-77eb-4ae4-b71a-8adac1ae7d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:42.934158 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee8e9b-77eb-4ae4-b71a-8adac1ae7d5b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:42.945075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee8e9b-77eb-4ae4-b71a-8adac1ae7d5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-bbee8e9b-77eb-4ae4-b71a-8adac1ae7d5b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:42.950182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:42.951129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:42.952040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:42.952040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:42.952040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:42.952040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:42.952040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:42.952040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:42.952040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:42.952040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:42.952040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:42.952040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:42.952040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 15:55:42.952040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:42.954774 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 464/928] 10.222.0.27 () {56 vars in 1120 bytes} [Thu Sep 8 15:55:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:42.960486 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb9c28-50a0-4b3f-be17-6b5fe42f59db None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:42.995279 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:43.062416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb9c28-50a0-4b3f-be17-6b5fe42f59db None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:43.097977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6389200b-2446-4ba7-9440-80385246c2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:43.098576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389200b-2446-4ba7-9440-80385246c2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:43.098848 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389200b-2446-4ba7-9440-80385246c2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:43.099110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6389200b-2446-4ba7-9440-80385246c2c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:43.099812 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389200b-2446-4ba7-9440-80385246c2c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:43.100108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389200b-2446-4ba7-9440-80385246c2c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:43.101539 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6389200b-2446-4ba7-9440-80385246c2c8 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:43.110521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 465/929] 10.222.0.27 () {56 vars in 1115 bytes} [Thu Sep 8 15:55:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:43.116464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee8e9b-77eb-4ae4-b71a-8adac1ae7d5b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:43.160323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:43.229876 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee8e9b-77eb-4ae4-b71a-8adac1ae7d5b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:43.264463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad719cc0-0e21-4bbc-a85f-489206106827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:43.265104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad719cc0-0e21-4bbc-a85f-489206106827 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:43.265375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad719cc0-0e21-4bbc-a85f-489206106827 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:43.265641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad719cc0-0e21-4bbc-a85f-489206106827 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:43.277836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad719cc0-0e21-4bbc-a85f-489206106827 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:43.279340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad719cc0-0e21-4bbc-a85f-489206106827 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-ad719cc0-0e21-4bbc-a85f-489206106827 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:43.297036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:43.298720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:43.301415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:43.301415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:43.301415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:43.301415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 465/930] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:43.305904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d836c-8903-4f9c-b107-63bfc862c092 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:43.306251 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d836c-8903-4f9c-b107-63bfc862c092 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:43.306473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d836c-8903-4f9c-b107-63bfc862c092 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:43.308091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 466/931] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:55:43.312352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad719cc0-0e21-4bbc-a85f-489206106827 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:43.313629 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9aee51-d2a4-4f7b-9f32-31f34da35bc2 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:43.313874 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9aee51-d2a4-4f7b-9f32-31f34da35bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:43.314209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9aee51-d2a4-4f7b-9f32-31f34da35bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:43.336065 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-0a9aee51-d2a4-4f7b-9f32-31f34da35bc2 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:43.337670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0a9aee51-d2a4-4f7b-9f32-31f34da35bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:43.414320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 467/932] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 15:55:43.415609 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c3f55-ba02-4a01-bab1-106d1740d864 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:43.415888 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c3f55-ba02-4a01-bab1-106d1740d864 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:43.416156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c3f55-ba02-4a01-bab1-106d1740d864 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:43.417624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 468/933] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:43.425530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f884a45-85f6-442a-9a80-4e4bd9ebe364 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:43.425807 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f884a45-85f6-442a-9a80-4e4bd9ebe364 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:43.426076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f884a45-85f6-442a-9a80-4e4bd9ebe364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:43.434393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:43.443249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad719cc0-0e21-4bbc-a85f-489206106827 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:43.447274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-6f884a45-85f6-442a-9a80-4e4bd9ebe364 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:43.448519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6f884a45-85f6-442a-9a80-4e4bd9ebe364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:43.475213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:43.485712 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee9a78-8fdf-46b9-9101-305ea42ccd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:43.486361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee9a78-8fdf-46b9-9101-305ea42ccd65 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:43.486644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee9a78-8fdf-46b9-9101-305ea42ccd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:43.486916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dee9a78-8fdf-46b9-9101-305ea42ccd65 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:43.487616 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee9a78-8fdf-46b9-9101-305ea42ccd65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:43.487943 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee9a78-8fdf-46b9-9101-305ea42ccd65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:43.489276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dee9a78-8fdf-46b9-9101-305ea42ccd65 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:43.506637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 466/934] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:43.508309 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee9a78-8fdf-46b9-9101-305ea42ccd65 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:43.531587 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 469/935] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:43.620696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dee9a78-8fdf-46b9-9101-305ea42ccd65 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:43.669201 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac5f04-a37c-4fce-bfbf-6995ed8403b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:43.670267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac5f04-a37c-4fce-bfbf-6995ed8403b9 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:43.670787 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac5f04-a37c-4fce-bfbf-6995ed8403b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:43.671258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac5f04-a37c-4fce-bfbf-6995ed8403b9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:43.680040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac5f04-a37c-4fce-bfbf-6995ed8403b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:43.682754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac5f04-a37c-4fce-bfbf-6995ed8403b9 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:43.688267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f884a45-85f6-442a-9a80-4e4bd9ebe364 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-4dac5f04-a37c-4fce-bfbf-6995ed8403b9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:43.697888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:43.698849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:43.699931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:43.699931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:43.699931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:43.699931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:43.699931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:43.699931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:43.699931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:43.699931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:43.699931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 15:55:43.699931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:43.702803 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 467/936] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Sep 8 15:55:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:43.710201 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac5f04-a37c-4fce-bfbf-6995ed8403b9 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:43.729160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:43.803779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f884a45-85f6-442a-9a80-4e4bd9ebe364 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:43.833349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac5f04-a37c-4fce-bfbf-6995ed8403b9 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:43.840956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07474a4e-25eb-4f78-abfe-a865e6faccc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:43.841598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07474a4e-25eb-4f78-abfe-a865e6faccc1 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:43.841873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07474a4e-25eb-4f78-abfe-a865e6faccc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:43.842144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07474a4e-25eb-4f78-abfe-a865e6faccc1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:43.842732 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07474a4e-25eb-4f78-abfe-a865e6faccc1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:43.844144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07474a4e-25eb-4f78-abfe-a865e6faccc1 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-07474a4e-25eb-4f78-abfe-a865e6faccc1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:43.855500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:43.856456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:43.857376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:43.857376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:43.857376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:43.857376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:43.857376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:43.857376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:43.857376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:43.857376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:43.857376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:43.857376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:43.857376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:43.857376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:43.857376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 15:55:43.857376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:43.857376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 470/937] 10.222.0.27 () {56 vars in 1132 bytes} [Thu Sep 8 15:55:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:43.863399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07474a4e-25eb-4f78-abfe-a865e6faccc1 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:43.873024 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e82afe-571a-4430-94b3-2ea1c8b3329e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:43.873588 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e82afe-571a-4430-94b3-2ea1c8b3329e None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:43.873858 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e82afe-571a-4430-94b3-2ea1c8b3329e None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:43.874130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e82afe-571a-4430-94b3-2ea1c8b3329e None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:43.874867 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e82afe-571a-4430-94b3-2ea1c8b3329e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:43.875174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e82afe-571a-4430-94b3-2ea1c8b3329e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:43.876692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e82afe-571a-4430-94b3-2ea1c8b3329e None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:43.889705 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:43.900552 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 468/938] 10.222.0.27 () {56 vars in 1149 bytes} [Thu Sep 8 15:55:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:43.900552 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e82afe-571a-4430-94b3-2ea1c8b3329e None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:43.965462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:43.984823 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07474a4e-25eb-4f78-abfe-a865e6faccc1 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:44.007947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e82afe-571a-4430-94b3-2ea1c8b3329e None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:44.022188 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee418d-409e-4708-a2de-c704124d5eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:44.022806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee418d-409e-4708-a2de-c704124d5eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:44.023077 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee418d-409e-4708-a2de-c704124d5eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:44.023342 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee418d-409e-4708-a2de-c704124d5eb8 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:44.024207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee418d-409e-4708-a2de-c704124d5eb8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:44.024618 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee418d-409e-4708-a2de-c704124d5eb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:44.026634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee418d-409e-4708-a2de-c704124d5eb8 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:44.032698 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 471/939] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Sep 8 15:55:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:44.033489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee418d-409e-4708-a2de-c704124d5eb8 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:44.056928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a1b42-7ec1-459e-ace6-75f72c6938a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:44.057554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a1b42-7ec1-459e-ace6-75f72c6938a5 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:44.057891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a1b42-7ec1-459e-ace6-75f72c6938a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:44.058163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a1b42-7ec1-459e-ace6-75f72c6938a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:44.058817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a1b42-7ec1-459e-ace6-75f72c6938a5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:44.060383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a1b42-7ec1-459e-ace6-75f72c6938a5 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:44.063160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: WARNING keystone.server.flask.common [None req-2e7a1b42-7ec1-459e-ace6-75f72c6938a5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 08 15:55:44.082174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.sql.core [None req-2e7a1b42-7ec1-459e-ace6-75f72c6938a5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 15:55:44.082174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 08 15:55:44.082174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [parameters: {'id': 'eee8559080b74a268eb8a8e02deb2aec', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 08 15:55:44.082174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81527) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 15:55:44.100363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-2e7a1b42-7ec1-459e-ace6-75f72c6938a5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eee8559080b74a268eb8a8e02deb2aec.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eee8559080b74a268eb8a8e02deb2aec. Sep 08 15:55:44.100363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:44.100363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 15:55:44.100363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 15:55:44.100363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 15:55:44.100363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 15:55:44.100363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 15:55:44.100363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 15:55:44.100363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 15:55:44.100363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application conn.query(q) Sep 08 15:55:44.100363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 15:55:44.100363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 15:55:44.100363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application result.read() Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 08 15:55:44.101282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application next(self.gen) Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application yield resource Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application next(self.gen) Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise self.value Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application session.commit() Sep 08 15:55:44.102267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self.session.flush() Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application compat.raise_( Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 15:55:44.103199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 15:55:44.104133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 15:55:44.104133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 15:55:44.104133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 15:55:44.104133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 15:55:44.104133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 08 15:55:44.104133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 08 15:55:44.104133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 15:55:44.104133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 15:55:44.104133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 15:55:44.104133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 15:55:44.104133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 15:55:44.104133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 15:55:44.104133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 15:55:44.104133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 15:55:44.104133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application conn.query(q) Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application result.read() Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 15:55:44.104944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 15:55:44.107650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 15:55:44.107650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 15:55:44.107650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 15:55:44.107650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 15:55:44.107650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 15:55:44.107650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 15:55:44.107650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 15:55:44.107650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 08 15:55:44.107650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [parameters: {'id': 'eee8559080b74a268eb8a8e02deb2aec', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 08 15:55:44.107650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 15:55:44.107650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:44.107650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:44.107650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:44.107650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:44.107650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:44.109875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:44.112924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:44.112924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 08 15:55:44.112924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 08 15:55:44.112924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:44.112924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:44.112924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 08 15:55:44.112924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 08 15:55:44.112924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eee8559080b74a268eb8a8e02deb2aec. Sep 08 15:55:44.112924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:44.112924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 469/940] 10.222.0.27 () {60 vars in 1140 bytes} [Thu Sep 8 15:55:43 2022] POST /identity/v3/projects => generated 262 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:44.112924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a1b42-7ec1-459e-ace6-75f72c6938a5 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:44.132821 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:44.150513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee418d-409e-4708-a2de-c704124d5eb8 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:44.188641 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ec199-afb2-46b4-b0b9-7e2b04fc9c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:44.189445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ec199-afb2-46b4-b0b9-7e2b04fc9c36 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:44.189717 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ec199-afb2-46b4-b0b9-7e2b04fc9c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:44.189982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ec199-afb2-46b4-b0b9-7e2b04fc9c36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:44.202143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ec199-afb2-46b4-b0b9-7e2b04fc9c36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:44.203647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ec199-afb2-46b4-b0b9-7e2b04fc9c36 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:44.218556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a1b42-7ec1-459e-ace6-75f72c6938a5 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-019ec199-afb2-46b4-b0b9-7e2b04fc9c36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:44.218839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:44.219795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:44.220794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:44.220794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:44.220794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:44.220794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 472/941] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:44.221580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ec199-afb2-46b4-b0b9-7e2b04fc9c36 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:44.248083 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:44.261871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3c6df-c554-40c0-aa02-f7ef9408fc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:44.262900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3c6df-c554-40c0-aa02-f7ef9408fc4f None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:44.263182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3c6df-c554-40c0-aa02-f7ef9408fc4f None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:44.263462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3c6df-c554-40c0-aa02-f7ef9408fc4f None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:44.264469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3c6df-c554-40c0-aa02-f7ef9408fc4f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:44.264783 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3c6df-c554-40c0-aa02-f7ef9408fc4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:44.266662 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3c6df-c554-40c0-aa02-f7ef9408fc4f None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:44.275435 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 470/942] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Sep 8 15:55:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:44.276190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3c6df-c554-40c0-aa02-f7ef9408fc4f None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:44.331431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ec199-afb2-46b4-b0b9-7e2b04fc9c36 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:44.370982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d68bb-c502-4ae6-a578-ff0b37b59beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:44.371840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d68bb-c502-4ae6-a578-ff0b37b59beb None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:44.372120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d68bb-c502-4ae6-a578-ff0b37b59beb None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:44.372685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d68bb-c502-4ae6-a578-ff0b37b59beb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:44.380522 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d68bb-c502-4ae6-a578-ff0b37b59beb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:44.381999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9d68bb-c502-4ae6-a578-ff0b37b59beb None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:44.384342 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:44.387862 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3c6df-c554-40c0-aa02-f7ef9408fc4f None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:44.399393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-ec9d68bb-c502-4ae6-a578-ff0b37b59beb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:44.399393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:44.399393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:44.399393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:44.399393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:44.399393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:44.399393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:44.399393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:44.399393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:44.399393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:44.399393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:44.399393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:44.399393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:44.399393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:44.399393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:44.400425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:44.401313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:44.401313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:44.401313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:44.401313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:44.401313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:44.401313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:44.401313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:44.401313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:44.401313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 473/943] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:44.402299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d68bb-c502-4ae6-a578-ff0b37b59beb None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:44.431507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89198783-21f0-4605-a9a7-a68be1e53cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:44.432130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89198783-21f0-4605-a9a7-a68be1e53cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:44.432543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89198783-21f0-4605-a9a7-a68be1e53cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:44.432818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89198783-21f0-4605-a9a7-a68be1e53cd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:44.433472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89198783-21f0-4605-a9a7-a68be1e53cd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:44.433779 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89198783-21f0-4605-a9a7-a68be1e53cd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:44.435284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89198783-21f0-4605-a9a7-a68be1e53cd7 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:44.448791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 471/944] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:44.449693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89198783-21f0-4605-a9a7-a68be1e53cd7 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:44.502633 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:44.512201 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9d68bb-c502-4ae6-a578-ff0b37b59beb None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:44.549658 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa118b0-ee35-41b1-a39d-e6bb36270940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:44.550333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa118b0-ee35-41b1-a39d-e6bb36270940 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:44.550685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa118b0-ee35-41b1-a39d-e6bb36270940 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:44.551021 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa118b0-ee35-41b1-a39d-e6bb36270940 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:44.551882 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa118b0-ee35-41b1-a39d-e6bb36270940 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:44.553468 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa118b0-ee35-41b1-a39d-e6bb36270940 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:44.555982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89198783-21f0-4605-a9a7-a68be1e53cd7 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:44.574196 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 474/945] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 15:55:44 2022] POST /identity/v3/endpoints => generated 347 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:55:44.575006 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa118b0-ee35-41b1-a39d-e6bb36270940 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:44.601296 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3fa41-10da-404c-b16f-364adfe106ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:44.601963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3fa41-10da-404c-b16f-364adfe106ef None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:44.602314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3fa41-10da-404c-b16f-364adfe106ef None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:44.602580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3fa41-10da-404c-b16f-364adfe106ef None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:44.603387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3fa41-10da-404c-b16f-364adfe106ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:44.603727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3fa41-10da-404c-b16f-364adfe106ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:44.605214 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3fa41-10da-404c-b16f-364adfe106ef None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:44.626124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 472/946] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Sep 8 15:55:44 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:44.627655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b959d97-e874-49ef-8655-c74f9cd69262 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:44.627939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b959d97-e874-49ef-8655-c74f9cd69262 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:44.628208 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b959d97-e874-49ef-8655-c74f9cd69262 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:44.631490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 473/947] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:44.636098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f08d49-10c3-4ef5-a715-9cc1e0f5846b None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:44.636383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f08d49-10c3-4ef5-a715-9cc1e0f5846b None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:44.636659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f08d49-10c3-4ef5-a715-9cc1e0f5846b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:44.648028 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:44.664327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-11f08d49-10c3-4ef5-a715-9cc1e0f5846b None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:44.667014 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-11f08d49-10c3-4ef5-a715-9cc1e0f5846b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:44.715496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa118b0-ee35-41b1-a39d-e6bb36270940 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:44.757907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6cb66-28c8-4ceb-9243-3633051e7c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:44.758929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6cb66-28c8-4ceb-9243-3633051e7c80 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:44.759522 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6cb66-28c8-4ceb-9243-3633051e7c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:44.759945 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6cb66-28c8-4ceb-9243-3633051e7c80 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:44.775489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 474/948] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:44.775727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6cb66-28c8-4ceb-9243-3633051e7c80 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:44.775727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6cb66-28c8-4ceb-9243-3633051e7c80 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:44.786506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-50a6cb66-28c8-4ceb-9243-3633051e7c80 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:44.786506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:44.786506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:44.786506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:44.786506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:44.786506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:44.786506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:44.786506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:44.786506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:44.786506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:44.786506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:44.786506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:44.786506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:44.786506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:44.786506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:44.787681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:44.788603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:44.788603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:44.788603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:44.788603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:44.788603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:44.788603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:44.788603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:44.788603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:44.788603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 475/949] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:44.795533 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f08d49-10c3-4ef5-a715-9cc1e0f5846b None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:44.832997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:44.905071 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f08d49-10c3-4ef5-a715-9cc1e0f5846b None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:44.942607 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c606229-9504-433d-8c8d-329a2114a858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:44.943384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c606229-9504-433d-8c8d-329a2114a858 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:44.943813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c606229-9504-433d-8c8d-329a2114a858 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:44.944175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c606229-9504-433d-8c8d-329a2114a858 None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:44.944973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c606229-9504-433d-8c8d-329a2114a858 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:44.945375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c606229-9504-433d-8c8d-329a2114a858 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:44.946942 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c606229-9504-433d-8c8d-329a2114a858 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:44.959616 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 475/950] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:44.965613 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6cb66-28c8-4ceb-9243-3633051e7c80 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:45.038771 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c606229-9504-433d-8c8d-329a2114a858 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:45.072244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6cb66-28c8-4ceb-9243-3633051e7c80 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:45.087620 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:45.111777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b216d3a-5435-4ca4-8f67-b019011c5da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:45.112455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b216d3a-5435-4ca4-8f67-b019011c5da5 None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:45.112729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b216d3a-5435-4ca4-8f67-b019011c5da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:45.112998 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b216d3a-5435-4ca4-8f67-b019011c5da5 None admin] PATH_INFO: `/v3/projects/a825a0b1238d4382995da440944f453b/users/b4324e4c04e343c98d9bb216c542fcba/roles/f752d851596f48a3bb97ad5d8f09d763` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:45.115643 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b216d3a-5435-4ca4-8f67-b019011c5da5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a825a0b1238d4382995da440944f453b, user_id=b4324e4c04e343c98d9bb216c542fcba, role_id=f752d851596f48a3bb97ad5d8f09d763)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:45.117123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b216d3a-5435-4ca4-8f67-b019011c5da5 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:45.137897 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 476/951] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:55:44 2022] PUT /identity/v3/projects/a825a0b1238d4382995da440944f453b/users/b4324e4c04e343c98d9bb216c542fcba/roles/f752d851596f48a3bb97ad5d8f09d763 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:55:45.138207 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:45.151038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c606229-9504-433d-8c8d-329a2114a858 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:45.190802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56da82-bffa-4589-b6a8-4240698da843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:45.191515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56da82-bffa-4589-b6a8-4240698da843 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:45.191798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56da82-bffa-4589-b6a8-4240698da843 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:45.192062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a56da82-bffa-4589-b6a8-4240698da843 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:45.192649 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56da82-bffa-4589-b6a8-4240698da843 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:45.194151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a56da82-bffa-4589-b6a8-4240698da843 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-8a56da82-bffa-4589-b6a8-4240698da843 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:45.204687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:45.205889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:45.206940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:45.206940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:45.206940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:45.206940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:45.206940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:45.206940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:45.206940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:45.206940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:45.206940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:45.206940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:45.206940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:45.206940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:45.206940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 15:55:45.206940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:45.206940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 476/952] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Sep 8 15:55:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:45.216663 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b216d3a-5435-4ca4-8f67-b019011c5da5 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:45.337382 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b216d3a-5435-4ca4-8f67-b019011c5da5 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:45.373444 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9ee3c-3e6b-4660-9f64-079e951b0e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:45.374078 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9ee3c-3e6b-4660-9f64-079e951b0e42 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:45.374357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9ee3c-3e6b-4660-9f64-079e951b0e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:45.374630 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e9ee3c-3e6b-4660-9f64-079e951b0e42 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:45.375277 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e9ee3c-3e6b-4660-9f64-079e951b0e42 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:45.375621 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e9ee3c-3e6b-4660-9f64-079e951b0e42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:45.377080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e9ee3c-3e6b-4660-9f64-079e951b0e42 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:45.382471 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 477/953] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Sep 8 15:55:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:45.388471 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56da82-bffa-4589-b6a8-4240698da843 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:45.497365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a56da82-bffa-4589-b6a8-4240698da843 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:45.533139 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693050c7-8686-43b3-9c0e-8cb9606d0d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:45.534152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693050c7-8686-43b3-9c0e-8cb9606d0d44 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:45.534622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693050c7-8686-43b3-9c0e-8cb9606d0d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:45.535081 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693050c7-8686-43b3-9c0e-8cb9606d0d44 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:45.535971 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693050c7-8686-43b3-9c0e-8cb9606d0d44 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:45.536465 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693050c7-8686-43b3-9c0e-8cb9606d0d44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:45.538121 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693050c7-8686-43b3-9c0e-8cb9606d0d44 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:45.543906 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 477/954] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Sep 8 15:55:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:45.550218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9ee3c-3e6b-4660-9f64-079e951b0e42 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:45.600570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:45.661893 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e9ee3c-3e6b-4660-9f64-079e951b0e42 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:45.697769 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf146c7e-cc10-4a4d-9ce5-76174ec6d58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:45.698410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf146c7e-cc10-4a4d-9ce5-76174ec6d58c None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:45.698688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf146c7e-cc10-4a4d-9ce5-76174ec6d58c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:45.698956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf146c7e-cc10-4a4d-9ce5-76174ec6d58c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:45.699703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf146c7e-cc10-4a4d-9ce5-76174ec6d58c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:45.701148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf146c7e-cc10-4a4d-9ce5-76174ec6d58c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:45.726458 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 478/955] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 15:55:45 2022] POST /identity/v3/endpoints => generated 349 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:55:46.887849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e47bc-61c1-49d5-8395-a0f46f10f550 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:46.888871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e47bc-61c1-49d5-8395-a0f46f10f550 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:46.889370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e47bc-61c1-49d5-8395-a0f46f10f550 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:46.891243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 478/956] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:46.896953 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ba813-f8bd-4457-92a6-fdfffbbb01d0 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:46.897788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ba813-f8bd-4457-92a6-fdfffbbb01d0 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:46.898278 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ba813-f8bd-4457-92a6-fdfffbbb01d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:46.917557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f27ba813-f8bd-4457-92a6-fdfffbbb01d0 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:46.919057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f27ba813-f8bd-4457-92a6-fdfffbbb01d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:46.969165 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e5925-5f58-47ab-a60f-2617b040f277 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:46.969443 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e5925-5f58-47ab-a60f-2617b040f277 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:46.969720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9e5925-5f58-47ab-a60f-2617b040f277 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:46.971391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 479/957] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:46.976652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c0315-3527-48ea-bdf2-cfa6727bdb42 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:46.976930 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c0315-3527-48ea-bdf2-cfa6727bdb42 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:46.977193 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c0315-3527-48ea-bdf2-cfa6727bdb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:46.995682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-4d8c0315-3527-48ea-bdf2-cfa6727bdb42 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:46.996866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4d8c0315-3527-48ea-bdf2-cfa6727bdb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:46.997544 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 479/958] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:47.018103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:47.068983 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 480/959] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 15:55:47.264486 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ba813-f8bd-4457-92a6-fdfffbbb01d0 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:47.289336 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:47.346075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c0315-3527-48ea-bdf2-cfa6727bdb42 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:47.388054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ba813-f8bd-4457-92a6-fdfffbbb01d0 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:47.434514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1d577-578e-4322-94f6-b6c140ecb292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:47.435133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1d577-578e-4322-94f6-b6c140ecb292 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:47.435661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1d577-578e-4322-94f6-b6c140ecb292 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:47.435960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1d577-578e-4322-94f6-b6c140ecb292 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:47.436576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1d577-578e-4322-94f6-b6c140ecb292 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:47.437969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1d577-578e-4322-94f6-b6c140ecb292 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-41f1d577-578e-4322-94f6-b6c140ecb292 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:47.450062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:47.451199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:47.452579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:47.452579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:47.452579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:47.452579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:47.452579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:47.452579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:47.452579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:47.452579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:47.452579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:47.452579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:47.452579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:47.452579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:47.452579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 15:55:47.452579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:47.452579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 480/960] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:47.453576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c7439-d853-493a-8179-ac7714dba486 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:47.458512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c7439-d853-493a-8179-ac7714dba486 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:47.458512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c7439-d853-493a-8179-ac7714dba486 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:47.458512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 481/961] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:47.460001 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c7439-d853-493a-8179-ac7714dba486 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:47.462212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c0315-3527-48ea-bdf2-cfa6727bdb42 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:47.501794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31413402-e927-40ed-b60f-e40035b20367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:47.502406 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31413402-e927-40ed-b60f-e40035b20367 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:47.502687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31413402-e927-40ed-b60f-e40035b20367 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:47.502956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31413402-e927-40ed-b60f-e40035b20367 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:47.503657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31413402-e927-40ed-b60f-e40035b20367 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:47.505089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31413402-e927-40ed-b60f-e40035b20367 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:47.520108 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.sql.core [None req-31413402-e927-40ed-b60f-e40035b20367 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 15:55:47.520108 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 08 15:55:47.520108 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [parameters: {'user_id': '11061aa96416410ab9b8d7d37c2042a3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 08 15:55:47.520108 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81527) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 15:55:47.527210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-31413402-e927-40ed-b60f-e40035b20367 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 08 15:55:47.527210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:47.527210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 15:55:47.527210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 15:55:47.527210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 15:55:47.527210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 15:55:47.527210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 15:55:47.527210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 15:55:47.527210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 15:55:47.527210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application conn.query(q) Sep 08 15:55:47.527210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 15:55:47.527210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 15:55:47.527210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 15:55:47.527210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application result.read() Sep 08 15:55:47.527210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 15:55:47.528128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application next(self.gen) Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application yield resource Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application next(self.gen) Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise self.value Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application session.commit() Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 15:55:47.529119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self.session.flush() Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application compat.raise_( Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 15:55:47.530425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 15:55:47.531328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application conn.query(q) Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application result.read() Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 15:55:47.532223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 15:55:47.533544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 15:55:47.533544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 15:55:47.533544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 15:55:47.533544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 15:55:47.533544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 15:55:47.533544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 08 15:55:47.533544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [parameters: {'user_id': '11061aa96416410ab9b8d7d37c2042a3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 08 15:55:47.533544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 15:55:47.533544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:47.533544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:47.533544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:47.533544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:47.533544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:47.533544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:47.533544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 08 15:55:47.534636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 08 15:55:47.536330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 08 15:55:47.536330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 08 15:55:47.536330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 15:55:47.536330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 15:55:47.536330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 08 15:55:47.536330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:47.536330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 481/962] 10.222.0.27 () {60 vars in 1131 bytes} [Thu Sep 8 15:55:47 2022] POST /identity/v3/users => generated 160 bytes in 183 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:47.536330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f322c67-ec34-4aae-a20e-fbcb764a8c43 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:47.536330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f322c67-ec34-4aae-a20e-fbcb764a8c43 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:47.536330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f322c67-ec34-4aae-a20e-fbcb764a8c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:47.550744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-0f322c67-ec34-4aae-a20e-fbcb764a8c43 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:47.552746 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0f322c67-ec34-4aae-a20e-fbcb764a8c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:47.564567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:47.564986 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:47.581765 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c7439-d853-493a-8179-ac7714dba486 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:47.614671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde75816-9b53-41ff-ac42-efd87e4b5d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:47.615325 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde75816-9b53-41ff-ac42-efd87e4b5d52 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:47.615636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde75816-9b53-41ff-ac42-efd87e4b5d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:47.615903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde75816-9b53-41ff-ac42-efd87e4b5d52 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:47.616537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde75816-9b53-41ff-ac42-efd87e4b5d52 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:47.616835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde75816-9b53-41ff-ac42-efd87e4b5d52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:47.618209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde75816-9b53-41ff-ac42-efd87e4b5d52 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:47.622703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 482/963] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Sep 8 15:55:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:47.623738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde75816-9b53-41ff-ac42-efd87e4b5d52 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:47.635561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 482/964] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:47.636718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f322c67-ec34-4aae-a20e-fbcb764a8c43 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:47.741594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde75816-9b53-41ff-ac42-efd87e4b5d52 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:47.758601 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f322c67-ec34-4aae-a20e-fbcb764a8c43 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:47.778272 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5b832-0bd8-4f85-8309-a49a32758727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:47.779257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5b832-0bd8-4f85-8309-a49a32758727 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:47.779756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5b832-0bd8-4f85-8309-a49a32758727 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:47.780320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5b832-0bd8-4f85-8309-a49a32758727 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:47.788802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5b832-0bd8-4f85-8309-a49a32758727 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:47.790778 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5b832-0bd8-4f85-8309-a49a32758727 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:47.799114 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6cc37e-bac0-4796-9fbe-47d6955ec711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:47.799782 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cc37e-bac0-4796-9fbe-47d6955ec711 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:47.800071 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cc37e-bac0-4796-9fbe-47d6955ec711 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:47.800321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cc37e-bac0-4796-9fbe-47d6955ec711 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:47.800967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6cc37e-bac0-4796-9fbe-47d6955ec711 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:47.801286 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6cc37e-bac0-4796-9fbe-47d6955ec711 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:47.802901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6cc37e-bac0-4796-9fbe-47d6955ec711 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-8ec5b832-0bd8-4f85-8309-a49a32758727 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:47.805540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:47.806527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:47.807534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:47.807534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:47.807534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:47.807534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:47.807534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:47.807534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:47.807534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:47.807534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:47.807534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 15:55:47.807534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:47.810506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 483/965] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:55:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:47.816216 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5b832-0bd8-4f85-8309-a49a32758727 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:47.817004 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 483/966] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Sep 8 15:55:47 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:47.832356 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:47.915586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6cc37e-bac0-4796-9fbe-47d6955ec711 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:47.929257 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:47.950560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5b832-0bd8-4f85-8309-a49a32758727 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:47.989729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe05cd-1972-4db2-a457-cc5d2ddf3893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:47.990700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe05cd-1972-4db2-a457-cc5d2ddf3893 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:47.991162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe05cd-1972-4db2-a457-cc5d2ddf3893 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:47.991792 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe05cd-1972-4db2-a457-cc5d2ddf3893 None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:47.992871 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe05cd-1972-4db2-a457-cc5d2ddf3893 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:47.993168 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe05cd-1972-4db2-a457-cc5d2ddf3893 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:47.994640 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe05cd-1972-4db2-a457-cc5d2ddf3893 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:48.031120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6cc37e-bac0-4796-9fbe-47d6955ec711 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:48.031330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 484/967] 10.222.0.27 () {56 vars in 1101 bytes} [Thu Sep 8 15:55:47 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:48.032367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bace77d-c275-4f8a-9b80-f6b801f3b4b7 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:48.032751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bace77d-c275-4f8a-9b80-f6b801f3b4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:48.033013 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bace77d-c275-4f8a-9b80-f6b801f3b4b7 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:48.034185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 485/968] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:48.040985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86866f-4dda-4bda-9e73-8dc1b70280b3 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:48.041264 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86866f-4dda-4bda-9e73-8dc1b70280b3 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:48.041543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d86866f-4dda-4bda-9e73-8dc1b70280b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:48.068748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-5d86866f-4dda-4bda-9e73-8dc1b70280b3 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:48.070285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5d86866f-4dda-4bda-9e73-8dc1b70280b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:48.080892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce56b7-910e-4c14-a75b-314d4ceb1fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:48.082190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce56b7-910e-4c14-a75b-314d4ceb1fad None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:48.082535 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce56b7-910e-4c14-a75b-314d4ceb1fad None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:48.082872 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce56b7-910e-4c14-a75b-314d4ceb1fad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:48.099001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce56b7-910e-4c14-a75b-314d4ceb1fad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-c9ce56b7-910e-4c14-a75b-314d4ceb1fad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:48.106670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:48.107973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:48.109044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:48.109044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:48.109044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:48.109044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:48.109044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:48.109044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:48.109044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:48.109044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:48.109044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:48.109044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:48.109044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 15:55:48.109044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:48.110446 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 484/969] 10.222.0.27 () {56 vars in 1120 bytes} [Thu Sep 8 15:55:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:48.118119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce56b7-910e-4c14-a75b-314d4ceb1fad None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:48.156599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:48.171486 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 486/970] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:48.216972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:48.257148 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce56b7-910e-4c14-a75b-314d4ceb1fad None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:48.293384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f7986-d57d-46e7-9684-3e1a444e32bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:48.294272 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f7986-d57d-46e7-9684-3e1a444e32bc None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:48.294714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f7986-d57d-46e7-9684-3e1a444e32bc None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:48.295132 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f7986-d57d-46e7-9684-3e1a444e32bc None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:48.296053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f7986-d57d-46e7-9684-3e1a444e32bc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:48.296500 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f7986-d57d-46e7-9684-3e1a444e32bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:48.298109 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91f7986-d57d-46e7-9684-3e1a444e32bc None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:48.307620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 485/971] 10.222.0.27 () {56 vars in 1115 bytes} [Thu Sep 8 15:55:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:48.313656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86866f-4dda-4bda-9e73-8dc1b70280b3 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:48.417982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d86866f-4dda-4bda-9e73-8dc1b70280b3 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:48.450071 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f7986-d57d-46e7-9684-3e1a444e32bc None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:48.452847 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980e319-61b6-486e-8552-ee0cbb09c1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:48.453796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980e319-61b6-486e-8552-ee0cbb09c1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:48.454147 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980e319-61b6-486e-8552-ee0cbb09c1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:48.454457 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980e319-61b6-486e-8552-ee0cbb09c1d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:48.465511 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980e319-61b6-486e-8552-ee0cbb09c1d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:48.466961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980e319-61b6-486e-8552-ee0cbb09c1d6 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-2980e319-61b6-486e-8552-ee0cbb09c1d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:48.481813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:48.482834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:48.488505 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:48.488505 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:48.488505 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:48.488505 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 487/972] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:48.489784 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980e319-61b6-486e-8552-ee0cbb09c1d6 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:48.544446 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:48.573160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f7986-d57d-46e7-9684-3e1a444e32bc None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:48.597387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980e319-61b6-486e-8552-ee0cbb09c1d6 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:48.605846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:48.613150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dd992-8a5f-4e89-8ce1-df954e7fe4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:48.613791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dd992-8a5f-4e89-8ce1-df954e7fe4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:48.614040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dd992-8a5f-4e89-8ce1-df954e7fe4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:48.614330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dd992-8a5f-4e89-8ce1-df954e7fe4c8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:48.614931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458dd992-8a5f-4e89-8ce1-df954e7fe4c8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:48.616334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458dd992-8a5f-4e89-8ce1-df954e7fe4c8 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-458dd992-8a5f-4e89-8ce1-df954e7fe4c8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:48.625567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:48.626664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:48.627565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:48.627565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:48.627565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:48.627565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:48.627565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:48.627565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:48.627565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:48.627565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:48.627565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:48.627565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:48.627565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:48.627565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:48.627565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 15:55:48.627565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:48.627565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 486/973] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Sep 8 15:55:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:48.632763 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dd992-8a5f-4e89-8ce1-df954e7fe4c8 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:48.638886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcd35f-2df2-4dfd-930a-d68ed23a699d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:48.639527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcd35f-2df2-4dfd-930a-d68ed23a699d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:48.639806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcd35f-2df2-4dfd-930a-d68ed23a699d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:48.640070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddcd35f-2df2-4dfd-930a-d68ed23a699d None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:48.640715 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddcd35f-2df2-4dfd-930a-d68ed23a699d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:48.641009 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddcd35f-2df2-4dfd-930a-d68ed23a699d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:48.642467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddcd35f-2df2-4dfd-930a-d68ed23a699d None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:48.654859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 488/974] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:48.660600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcd35f-2df2-4dfd-930a-d68ed23a699d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:48.748552 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dd992-8a5f-4e89-8ce1-df954e7fe4c8 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:48.763118 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddcd35f-2df2-4dfd-930a-d68ed23a699d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:48.775527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:48.789457 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f505cd8-a535-4aba-a5bc-2f2e7d404c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:48.790083 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f505cd8-a535-4aba-a5bc-2f2e7d404c41 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:48.790352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f505cd8-a535-4aba-a5bc-2f2e7d404c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:48.790620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f505cd8-a535-4aba-a5bc-2f2e7d404c41 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:48.791262 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f505cd8-a535-4aba-a5bc-2f2e7d404c41 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:48.791594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f505cd8-a535-4aba-a5bc-2f2e7d404c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:48.793009 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f505cd8-a535-4aba-a5bc-2f2e7d404c41 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:48.798109 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 487/975] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Sep 8 15:55:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:48.803693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f505cd8-a535-4aba-a5bc-2f2e7d404c41 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:48.805659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4bc49-c8f9-4be8-9789-385f75dcd514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:48.806290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4bc49-c8f9-4be8-9789-385f75dcd514 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:48.806562 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4bc49-c8f9-4be8-9789-385f75dcd514 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:48.806823 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4bc49-c8f9-4be8-9789-385f75dcd514 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:48.814804 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4bc49-c8f9-4be8-9789-385f75dcd514 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:48.816279 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4bc49-c8f9-4be8-9789-385f75dcd514 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-33f4bc49-c8f9-4be8-9789-385f75dcd514 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:55:48.827625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:48.828665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:48.829680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:48.829680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:48.829680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:55:48.829680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:55:48.829680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:55:48.829680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:55:48.829680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:55:48.829680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:55:48.829680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 15:55:48.829680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:48.829680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 489/976] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Sep 8 15:55:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:48.834611 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4bc49-c8f9-4be8-9789-385f75dcd514 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:48.871842 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:48.916823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f505cd8-a535-4aba-a5bc-2f2e7d404c41 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:48.941071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4bc49-c8f9-4be8-9789-385f75dcd514 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:48.952968 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783131f6-ee0e-479e-a804-79afd1518439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:48.953576 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783131f6-ee0e-479e-a804-79afd1518439 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:48.953840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783131f6-ee0e-479e-a804-79afd1518439 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:48.954264 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783131f6-ee0e-479e-a804-79afd1518439 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:48.955371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783131f6-ee0e-479e-a804-79afd1518439 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:48.955705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783131f6-ee0e-479e-a804-79afd1518439 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:48.957218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783131f6-ee0e-479e-a804-79afd1518439 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:48.963136 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 488/977] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Sep 8 15:55:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:48.968926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783131f6-ee0e-479e-a804-79afd1518439 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:48.976179 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82341964-f386-4c76-b4c4-657daa7cc311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:48.976793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82341964-f386-4c76-b4c4-657daa7cc311 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:48.977057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82341964-f386-4c76-b4c4-657daa7cc311 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:48.977733 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82341964-f386-4c76-b4c4-657daa7cc311 None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:48.978462 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82341964-f386-4c76-b4c4-657daa7cc311 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:48.978759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82341964-f386-4c76-b4c4-657daa7cc311 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:48.980872 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82341964-f386-4c76-b4c4-657daa7cc311 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:48.999371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 490/978] 10.222.0.27 () {56 vars in 1149 bytes} [Thu Sep 8 15:55:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:49.005569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82341964-f386-4c76-b4c4-657daa7cc311 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:49.035138 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:49.079676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783131f6-ee0e-479e-a804-79afd1518439 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:49.103703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:49.112742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82341964-f386-4c76-b4c4-657daa7cc311 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:49.118149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759bdbb-f450-4d89-84c7-a5c192efe4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:49.118767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759bdbb-f450-4d89-84c7-a5c192efe4aa None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:49.119039 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759bdbb-f450-4d89-84c7-a5c192efe4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:49.119303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9759bdbb-f450-4d89-84c7-a5c192efe4aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:49.120041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759bdbb-f450-4d89-84c7-a5c192efe4aa None admin] RBAC: Adding query filter params (interface=admin, service_id=5d6784bcb33a48eaafdbdd21c3435368, region_id=RegionOne) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:49.120724 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759bdbb-f450-4d89-84c7-a5c192efe4aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:49.122113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9759bdbb-f450-4d89-84c7-a5c192efe4aa None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:49.132907 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 489/979] 10.222.0.27 () {56 vars in 1249 bytes} [Thu Sep 8 15:55:48 2022] GET /identity/v3/endpoints?service_id=5d6784bcb33a48eaafdbdd21c3435368&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:55:49.133687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759bdbb-f450-4d89-84c7-a5c192efe4aa None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:49.150967 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d20d31f-617e-49b4-8ba1-20ed83e1a3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:49.151742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20d31f-617e-49b4-8ba1-20ed83e1a3ec None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:49.152302 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20d31f-617e-49b4-8ba1-20ed83e1a3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:49.152573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20d31f-617e-49b4-8ba1-20ed83e1a3ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:49.237480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d20d31f-617e-49b4-8ba1-20ed83e1a3ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:49.238989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d20d31f-617e-49b4-8ba1-20ed83e1a3ec None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:49.245820 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9759bdbb-f450-4d89-84c7-a5c192efe4aa None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:49.266433 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:49.280600 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abab76-05bc-4348-9aed-acc692a3a8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:49.281208 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abab76-05bc-4348-9aed-acc692a3a8e3 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:49.281483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abab76-05bc-4348-9aed-acc692a3a8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:49.281731 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abab76-05bc-4348-9aed-acc692a3a8e3 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:49.282333 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abab76-05bc-4348-9aed-acc692a3a8e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:49.283795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abab76-05bc-4348-9aed-acc692a3a8e3 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:49.298209 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 490/980] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Sep 8 15:55:49 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 15:55:49.346807 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 491/981] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:55:49 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:55:49.352821 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abab76-05bc-4348-9aed-acc692a3a8e3 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:49.406270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:49.459875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abab76-05bc-4348-9aed-acc692a3a8e3 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:49.495423 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee25b64-e606-488a-88f4-1e5f0fc258d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:49.496090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee25b64-e606-488a-88f4-1e5f0fc258d8 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:49.496386 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee25b64-e606-488a-88f4-1e5f0fc258d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:49.496649 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee25b64-e606-488a-88f4-1e5f0fc258d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:49.507843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee25b64-e606-488a-88f4-1e5f0fc258d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:49.509294 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee25b64-e606-488a-88f4-1e5f0fc258d8 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-8ee25b64-e606-488a-88f4-1e5f0fc258d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:55:49.523902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:49.524895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:55:49.525803 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:55:49.525803 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:55:49.525803 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:49.525803 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 491/982] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:49.530696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d20d31f-617e-49b4-8ba1-20ed83e1a3ec None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:49.639180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d20d31f-617e-49b4-8ba1-20ed83e1a3ec None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:49.665951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:49.676546 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cf7fe-ba43-42d8-8c85-92fec4298c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:49.677173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cf7fe-ba43-42d8-8c85-92fec4298c28 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:49.677448 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cf7fe-ba43-42d8-8c85-92fec4298c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:49.677717 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cf7fe-ba43-42d8-8c85-92fec4298c28 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:49.678373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9cf7fe-ba43-42d8-8c85-92fec4298c28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:49.678674 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9cf7fe-ba43-42d8-8c85-92fec4298c28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:49.680097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9cf7fe-ba43-42d8-8c85-92fec4298c28 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:49.692080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 492/983] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:49.697612 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee25b64-e606-488a-88f4-1e5f0fc258d8 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:49.799721 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee25b64-e606-488a-88f4-1e5f0fc258d8 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:49.801267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:49.835336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18018f6a-0f14-4808-83c0-db0a58785c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:49.836351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18018f6a-0f14-4808-83c0-db0a58785c32 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:49.836644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18018f6a-0f14-4808-83c0-db0a58785c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:49.836914 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18018f6a-0f14-4808-83c0-db0a58785c32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:49.843838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18018f6a-0f14-4808-83c0-db0a58785c32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:49.845337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18018f6a-0f14-4808-83c0-db0a58785c32 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:49.856851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-18018f6a-0f14-4808-83c0-db0a58785c32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:49.856851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:49.856851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:49.856851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:49.856851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:49.856851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:49.856851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:49.856851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:49.856851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:49.856851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:49.856851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:49.856851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:49.856851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:55:49.856851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:55:49.856851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:49.857882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:49.858784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:55:49.858784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:55:49.858784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:55:49.858784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:55:49.858784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:55:49.858784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:55:49.858784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:55:49.858784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:49.863485 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 492/984] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:49.864076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cf7fe-ba43-42d8-8c85-92fec4298c28 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:49.977237 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cf7fe-ba43-42d8-8c85-92fec4298c28 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:50.010710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2870e64-6a53-484f-9d76-c2a995766033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:50.011322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2870e64-6a53-484f-9d76-c2a995766033 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:50.011624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2870e64-6a53-484f-9d76-c2a995766033 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:50.011886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2870e64-6a53-484f-9d76-c2a995766033 None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:50.012569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2870e64-6a53-484f-9d76-c2a995766033 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:50.012867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2870e64-6a53-484f-9d76-c2a995766033 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:50.014308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2870e64-6a53-484f-9d76-c2a995766033 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:50.026909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 493/985] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:55:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:50.033182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18018f6a-0f14-4808-83c0-db0a58785c32 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:50.040470 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645bba6-e273-4a1e-9e4c-79523b8804d4 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:50.040757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645bba6-e273-4a1e-9e4c-79523b8804d4 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:50.041031 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f645bba6-e273-4a1e-9e4c-79523b8804d4 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:50.042531 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 494/986] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:55:50.047411 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaaef07-3c6d-42a7-9a81-c12385a45401 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:50.047718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaaef07-3c6d-42a7-9a81-c12385a45401 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:50.047977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaaef07-3c6d-42a7-9a81-c12385a45401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:50.068250 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-ccaaef07-3c6d-42a7-9a81-c12385a45401 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:50.069563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ccaaef07-3c6d-42a7-9a81-c12385a45401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:50.114561 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:50.138883 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18018f6a-0f14-4808-83c0-db0a58785c32 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:50.143366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 495/987] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 15:55:50.175109 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc60f40-4839-4f07-aefb-8a220a046389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:50.175930 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc60f40-4839-4f07-aefb-8a220a046389 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:50.176297 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc60f40-4839-4f07-aefb-8a220a046389 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:50.176657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc60f40-4839-4f07-aefb-8a220a046389 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:50.177519 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc60f40-4839-4f07-aefb-8a220a046389 None admin] RBAC: Adding query filter params (role.id=f752d851596f48a3bb97ad5d8f09d763, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=b4324e4c04e343c98d9bb216c542fcba) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:50.177926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc60f40-4839-4f07-aefb-8a220a046389 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:50.181449 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc60f40-4839-4f07-aefb-8a220a046389 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:50.195494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 493/988] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:55:50 2022] GET /identity/v3/role_assignments?user.id=b4324e4c04e343c98d9bb216c542fcba&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=f752d851596f48a3bb97ad5d8f09d763 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:50.405593 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaaef07-3c6d-42a7-9a81-c12385a45401 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:50.507743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaaef07-3c6d-42a7-9a81-c12385a45401 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:50.542782 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49066069-408f-4aa0-80ac-caf2ae8383d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:50.543416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49066069-408f-4aa0-80ac-caf2ae8383d1 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:50.543703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49066069-408f-4aa0-80ac-caf2ae8383d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:50.543968 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49066069-408f-4aa0-80ac-caf2ae8383d1 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:50.544641 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49066069-408f-4aa0-80ac-caf2ae8383d1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:50.546011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49066069-408f-4aa0-80ac-caf2ae8383d1 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:50.567490 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 496/989] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Sep 8 15:55:50 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:55:51.472219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bd245-157c-4873-a9f8-6c86e32b40b1 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:51.472744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bd245-157c-4873-a9f8-6c86e32b40b1 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:51.472744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bd245-157c-4873-a9f8-6c86e32b40b1 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:51.474078 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 494/990] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:51.479469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dd017-1eb7-4d00-8a5e-9e66ec7ecb98 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:51.479919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dd017-1eb7-4d00-8a5e-9e66ec7ecb98 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:51.480009 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dd017-1eb7-4d00-8a5e-9e66ec7ecb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:51.498897 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-5e1dd017-1eb7-4d00-8a5e-9e66ec7ecb98 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:51.500105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5e1dd017-1eb7-4d00-8a5e-9e66ec7ecb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:51.530559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:51.581795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 497/991] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:51.845754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bd245-157c-4873-a9f8-6c86e32b40b1 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:51.874579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:51.951153 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bd245-157c-4873-a9f8-6c86e32b40b1 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:51.986258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875bdf0-8d3e-4da4-876f-ccb310e05c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:51.986896 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875bdf0-8d3e-4da4-876f-ccb310e05c7a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:51.987175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875bdf0-8d3e-4da4-876f-ccb310e05c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:51.987413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8875bdf0-8d3e-4da4-876f-ccb310e05c7a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:51.988045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875bdf0-8d3e-4da4-876f-ccb310e05c7a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:51.989425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8875bdf0-8d3e-4da4-876f-ccb310e05c7a None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-8875bdf0-8d3e-4da4-876f-ccb310e05c7a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:51.998498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:51.999554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:52.000477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:52.000477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:52.000477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:52.000477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:52.000477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:52.000477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:52.000477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:52.000477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:52.000477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:52.000477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:52.000477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:52.000477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:52.000477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 15:55:52.000477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:52.001916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 495/992] 10.222.0.27 () {56 vars in 1126 bytes} [Thu Sep 8 15:55:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:52.005641 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dd017-1eb7-4d00-8a5e-9e66ec7ecb98 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:52.110733 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dd017-1eb7-4d00-8a5e-9e66ec7ecb98 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:52.146711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f09580-89b6-4e78-8146-86803694345c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:52.147327 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f09580-89b6-4e78-8146-86803694345c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:52.147633 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f09580-89b6-4e78-8146-86803694345c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:52.147901 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f09580-89b6-4e78-8146-86803694345c None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:52.148540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f09580-89b6-4e78-8146-86803694345c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:52.148840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f09580-89b6-4e78-8146-86803694345c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:52.150223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f09580-89b6-4e78-8146-86803694345c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:52.159656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 498/993] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Sep 8 15:55:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:52.161541 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875bdf0-8d3e-4da4-876f-ccb310e05c7a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:52.273134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:52.283602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8875bdf0-8d3e-4da4-876f-ccb310e05c7a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:52.320462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a13501-c368-40a0-a75f-8f357b88732f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:52.321080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a13501-c368-40a0-a75f-8f357b88732f None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:52.321357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a13501-c368-40a0-a75f-8f357b88732f None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:52.321626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a13501-c368-40a0-a75f-8f357b88732f None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:52.322260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a13501-c368-40a0-a75f-8f357b88732f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:52.322621 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a13501-c368-40a0-a75f-8f357b88732f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:52.324037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a13501-c368-40a0-a75f-8f357b88732f None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:52.327764 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59366095-dc33-4421-9ff8-c78eacffc10a None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:52.328139 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59366095-dc33-4421-9ff8-c78eacffc10a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:52.328480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59366095-dc33-4421-9ff8-c78eacffc10a None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:52.329448 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 496/994] 10.222.0.27 () {56 vars in 1122 bytes} [Thu Sep 8 15:55:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:52.330057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 499/995] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:52.335394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b532ea-430b-494f-afb5-079b1d5c2e86 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:52.335681 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b532ea-430b-494f-afb5-079b1d5c2e86 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:52.335785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59366095-dc33-4421-9ff8-c78eacffc10a None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:52.336318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b532ea-430b-494f-afb5-079b1d5c2e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:52.359571 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-48b532ea-430b-494f-afb5-079b1d5c2e86 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:52.361382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-48b532ea-430b-494f-afb5-079b1d5c2e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:52.366089 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:52.432041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 497/996] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:52.451979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59366095-dc33-4421-9ff8-c78eacffc10a None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:52.487921 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667f405-852f-4209-9443-f9c48eac5af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:52.488524 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667f405-852f-4209-9443-f9c48eac5af7 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:52.488788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667f405-852f-4209-9443-f9c48eac5af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:52.489058 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667f405-852f-4209-9443-f9c48eac5af7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:52.489724 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667f405-852f-4209-9443-f9c48eac5af7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:52.491096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667f405-852f-4209-9443-f9c48eac5af7 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:52.508588 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 500/997] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 15:55:52 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:55:52.696685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b532ea-430b-494f-afb5-079b1d5c2e86 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:52.703325 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38218a2c-6fae-46eb-8ae4-89b49e22ac0a None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:52.703624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38218a2c-6fae-46eb-8ae4-89b49e22ac0a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:52.703891 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38218a2c-6fae-46eb-8ae4-89b49e22ac0a None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:52.705860 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 501/998] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:52.711684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd97e0-8a0f-4789-9b91-290cb9113a12 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:52.711957 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd97e0-8a0f-4789-9b91-290cb9113a12 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:52.712221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd97e0-8a0f-4789-9b91-290cb9113a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:52.731922 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-dcdd97e0-8a0f-4789-9b91-290cb9113a12 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:52.734375 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-dcdd97e0-8a0f-4789-9b91-290cb9113a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:52.790901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:52.809362 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b532ea-430b-494f-afb5-079b1d5c2e86 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:52.811614 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 502/999] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:52 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 15:55:52.846431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64712ae-35b4-4a61-87d9-6ec14b209e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:52.847522 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64712ae-35b4-4a61-87d9-6ec14b209e29 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:52.848008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64712ae-35b4-4a61-87d9-6ec14b209e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:52.848478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64712ae-35b4-4a61-87d9-6ec14b209e29 None admin] PATH_INFO: `/v3/services/image` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:52.849295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64712ae-35b4-4a61-87d9-6ec14b209e29 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:52.850880 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64712ae-35b4-4a61-87d9-6ec14b209e29 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-c64712ae-35b4-4a61-87d9-6ec14b209e29 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:52.860110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:52.861376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:52.862378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:52.862378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:52.862378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:52.862378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:52.862378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:52.862378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:52.862378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:52.862378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:52.862378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:52.862378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:52.862378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:52.862378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:52.862378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 15:55:52.862378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:52.864501 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 498/1000] 10.222.0.27 () {56 vars in 1105 bytes} [Thu Sep 8 15:55:52 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:52.870739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd97e0-8a0f-4789-9b91-290cb9113a12 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:52.885957 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:52.976627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd97e0-8a0f-4789-9b91-290cb9113a12 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:53.011657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41786a5e-ccfd-4694-9dc7-0430580e74bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:53.012269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41786a5e-ccfd-4694-9dc7-0430580e74bf None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:53.012536 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41786a5e-ccfd-4694-9dc7-0430580e74bf None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:53.012798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41786a5e-ccfd-4694-9dc7-0430580e74bf None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:53.013434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41786a5e-ccfd-4694-9dc7-0430580e74bf None admin] RBAC: Adding query filter params (name=image) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:53.013730 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41786a5e-ccfd-4694-9dc7-0430580e74bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:53.015106 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41786a5e-ccfd-4694-9dc7-0430580e74bf None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:53.023496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 503/1001] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:53.029502 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64712ae-35b4-4a61-87d9-6ec14b209e29 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:53.074254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41786a5e-ccfd-4694-9dc7-0430580e74bf None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:53.137705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64712ae-35b4-4a61-87d9-6ec14b209e29 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:53.159236 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:53.173999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a42fb-c7fc-483a-aed8-100ce2ad2abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:53.174623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a42fb-c7fc-483a-aed8-100ce2ad2abd None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:53.174904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a42fb-c7fc-483a-aed8-100ce2ad2abd None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:53.175171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a42fb-c7fc-483a-aed8-100ce2ad2abd None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:53.175846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a42fb-c7fc-483a-aed8-100ce2ad2abd None admin] RBAC: Adding query filter params (type=image) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:53.176144 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a42fb-c7fc-483a-aed8-100ce2ad2abd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:53.177701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79a42fb-c7fc-483a-aed8-100ce2ad2abd None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:53.192220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 499/1002] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:53 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:53.192318 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41786a5e-ccfd-4694-9dc7-0430580e74bf None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:53.225483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc329bb1-7c74-4b4e-8478-29584bd1a87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:53.226142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc329bb1-7c74-4b4e-8478-29584bd1a87c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:53.226419 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc329bb1-7c74-4b4e-8478-29584bd1a87c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:53.226749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc329bb1-7c74-4b4e-8478-29584bd1a87c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:53.227285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc329bb1-7c74-4b4e-8478-29584bd1a87c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:53.228728 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc329bb1-7c74-4b4e-8478-29584bd1a87c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-dc329bb1-7c74-4b4e-8478-29584bd1a87c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:53.238451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:53.239554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:53.240862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:53.240862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:53.240862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:53.240862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:53.240862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:53.240862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:53.240862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:53.240862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:53.240862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:53.240862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:53.240862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:53.240862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:53.240862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 15:55:53.240862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:53.240862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 504/1003] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:55:53.245955 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a42fb-c7fc-483a-aed8-100ce2ad2abd None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:53.263566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:53.359712 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79a42fb-c7fc-483a-aed8-100ce2ad2abd None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:53.397832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153a4db-4af2-40b0-8158-27c312ab8ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:53.398452 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153a4db-4af2-40b0-8158-27c312ab8ded None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:53.398728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153a4db-4af2-40b0-8158-27c312ab8ded None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:53.398996 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153a4db-4af2-40b0-8158-27c312ab8ded None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:53.399659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153a4db-4af2-40b0-8158-27c312ab8ded None admin] RBAC: Adding query filter params (name=compute) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:53.399962 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153a4db-4af2-40b0-8158-27c312ab8ded None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:53.401339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153a4db-4af2-40b0-8158-27c312ab8ded None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:53.406667 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 500/1004] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Sep 8 15:55:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:53.412660 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc329bb1-7c74-4b4e-8478-29584bd1a87c None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:53.518577 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc329bb1-7c74-4b4e-8478-29584bd1a87c None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:53.554699 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efae428-61cf-4c10-b43d-89b26da722e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:53.555369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efae428-61cf-4c10-b43d-89b26da722e4 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:53.555636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efae428-61cf-4c10-b43d-89b26da722e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:53.555919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efae428-61cf-4c10-b43d-89b26da722e4 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:53.556701 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efae428-61cf-4c10-b43d-89b26da722e4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:53.557000 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efae428-61cf-4c10-b43d-89b26da722e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:53.558382 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efae428-61cf-4c10-b43d-89b26da722e4 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:53.567531 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 505/1005] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Sep 8 15:55:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:53.569881 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153a4db-4af2-40b0-8158-27c312ab8ded None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:53.650240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:53.677843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153a4db-4af2-40b0-8158-27c312ab8ded None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:53.712254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea49b6-0e37-4720-8baf-f0ce62fe56ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:53.712915 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea49b6-0e37-4720-8baf-f0ce62fe56ab None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:53.713127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea49b6-0e37-4720-8baf-f0ce62fe56ab None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:53.713401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea49b6-0e37-4720-8baf-f0ce62fe56ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:53.714103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea49b6-0e37-4720-8baf-f0ce62fe56ab None admin] RBAC: Adding query filter params (interface=public, service_id=a47f822969dd4f8aa78227cf5864575b, region_id=RegionOne) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:53.714411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea49b6-0e37-4720-8baf-f0ce62fe56ab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:53.715815 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea49b6-0e37-4720-8baf-f0ce62fe56ab None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:53.721421 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 501/1006] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Sep 8 15:55:53 2022] GET /identity/v3/endpoints?service_id=a47f822969dd4f8aa78227cf5864575b&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:53.727286 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efae428-61cf-4c10-b43d-89b26da722e4 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:53.731340 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:53.838780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efae428-61cf-4c10-b43d-89b26da722e4 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:53.874746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e07a7-9470-4f4a-b9e6-072d02639c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:53.875655 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e07a7-9470-4f4a-b9e6-072d02639c0b None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:53.876081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e07a7-9470-4f4a-b9e6-072d02639c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:53.876564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e07a7-9470-4f4a-b9e6-072d02639c0b None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:53.877376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e07a7-9470-4f4a-b9e6-072d02639c0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:53.878981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e07a7-9470-4f4a-b9e6-072d02639c0b None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:53.886085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 506/1007] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Sep 8 15:55:53 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 15:55:54.689190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2755d-b351-4677-9865-867b4d7149d3 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:54.689441 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2755d-b351-4677-9865-867b4d7149d3 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:54.689949 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2755d-b351-4677-9865-867b4d7149d3 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:54.691505 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 502/1008] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:54.697991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd7009-c65d-4fd7-97b0-b2b1950e23b6 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:54.697991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd7009-c65d-4fd7-97b0-b2b1950e23b6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:54.697991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd7009-c65d-4fd7-97b0-b2b1950e23b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:54.717646 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-14fd7009-c65d-4fd7-97b0-b2b1950e23b6 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:54.719148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-14fd7009-c65d-4fd7-97b0-b2b1950e23b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:54.787296 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:54.795143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 507/1009] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:54 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:55.062092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2755d-b351-4677-9865-867b4d7149d3 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:55.172219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2755d-b351-4677-9865-867b4d7149d3 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:55.213089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeeeceb-be3a-4cab-b485-4b7d03b5e9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:55.213752 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeeeceb-be3a-4cab-b485-4b7d03b5e9ee None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:55.214033 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeeeceb-be3a-4cab-b485-4b7d03b5e9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:55.214275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeeeceb-be3a-4cab-b485-4b7d03b5e9ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:55.215520 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeeeceb-be3a-4cab-b485-4b7d03b5e9ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:55.217018 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeeeceb-be3a-4cab-b485-4b7d03b5e9ee None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:55.218910 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 503/1010] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:55.224929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd7009-c65d-4fd7-97b0-b2b1950e23b6 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:55.319987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8e3de-4af1-4709-87a3-7b297757745f None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:55.320185 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8e3de-4af1-4709-87a3-7b297757745f None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:55.320611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8e3de-4af1-4709-87a3-7b297757745f None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:55.322146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 504/1011] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:55.327843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599fa72-6869-4484-9c89-f18574176f37 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:55.328356 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599fa72-6869-4484-9c89-f18574176f37 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:55.328847 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599fa72-6869-4484-9c89-f18574176f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:55.331922 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd7009-c65d-4fd7-97b0-b2b1950e23b6 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:55.348000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-6599fa72-6869-4484-9c89-f18574176f37 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:55.349475 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6599fa72-6869-4484-9c89-f18574176f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:55.351859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:55.368875 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f19ab-881a-4e55-af77-311b90dcbae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:55.369462 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f19ab-881a-4e55-af77-311b90dcbae0 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:55.369738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f19ab-881a-4e55-af77-311b90dcbae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:55.370007 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534f19ab-881a-4e55-af77-311b90dcbae0 None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:55.370667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f19ab-881a-4e55-af77-311b90dcbae0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:55.372142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534f19ab-881a-4e55-af77-311b90dcbae0 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:55.399770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 508/1012] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Sep 8 15:55:55 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:55:55.427489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 505/1013] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:55 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:55.690730 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f19ab-881a-4e55-af77-311b90dcbae0 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:55.743586 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:55.797637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534f19ab-881a-4e55-af77-311b90dcbae0 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:55.833466 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05578f32-619f-4ae0-989c-619dfca67615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:55.834105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05578f32-619f-4ae0-989c-619dfca67615 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:55.834381 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05578f32-619f-4ae0-989c-619dfca67615 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:55.834650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05578f32-619f-4ae0-989c-619dfca67615 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:55.835314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05578f32-619f-4ae0-989c-619dfca67615 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:55.837137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05578f32-619f-4ae0-989c-619dfca67615 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:55.855477 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 509/1014] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Sep 8 15:55:55 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:55:56.072255 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a0166-0929-4722-aecb-c76e2620c27c None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:56.072779 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a0166-0929-4722-aecb-c76e2620c27c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:56.072858 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a0166-0929-4722-aecb-c76e2620c27c None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:56.074176 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 506/1015] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:56.079898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fdba4-fcfe-46f6-8555-133cab07b16e None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:56.080190 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fdba4-fcfe-46f6-8555-133cab07b16e None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:56.080462 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fdba4-fcfe-46f6-8555-133cab07b16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:56.101267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-5d2fdba4-fcfe-46f6-8555-133cab07b16e None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:56.102476 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5d2fdba4-fcfe-46f6-8555-133cab07b16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:56.186334 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 510/1016] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:56 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:56.452359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a0166-0929-4722-aecb-c76e2620c27c None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:56.559745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a0166-0929-4722-aecb-c76e2620c27c None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:56.595071 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd0d83-858d-4e5d-aa43-1e79014f3250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:56.595730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd0d83-858d-4e5d-aa43-1e79014f3250 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:56.596000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd0d83-858d-4e5d-aa43-1e79014f3250 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:56.596264 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd0d83-858d-4e5d-aa43-1e79014f3250 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:56.596842 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd0d83-858d-4e5d-aa43-1e79014f3250 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:56.598184 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd0d83-858d-4e5d-aa43-1e79014f3250 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-20dd0d83-858d-4e5d-aa43-1e79014f3250 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:56.606980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:56.608061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:56.612589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:56.612589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:56.612589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:56.612589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:56.612589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:56.612589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:56.612589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:56.612589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:56.612589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:56.612589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:56.612589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:56.612589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:56.612589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 15:55:56.612589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:56.612589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 507/1017] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:55:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:56.615493 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fdba4-fcfe-46f6-8555-133cab07b16e None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:56.652280 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:56.720857 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fdba4-fcfe-46f6-8555-133cab07b16e None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:56.755179 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cccd4d-466a-41ec-9c4a-7bb779d025f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:56.755870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cccd4d-466a-41ec-9c4a-7bb779d025f2 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:56.756121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cccd4d-466a-41ec-9c4a-7bb779d025f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:56.756355 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cccd4d-466a-41ec-9c4a-7bb779d025f2 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:56.756989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cccd4d-466a-41ec-9c4a-7bb779d025f2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:56.757293 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cccd4d-466a-41ec-9c4a-7bb779d025f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:56.758667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cccd4d-466a-41ec-9c4a-7bb779d025f2 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:56.768634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 511/1018] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Sep 8 15:55:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:56.770230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd0d83-858d-4e5d-aa43-1e79014f3250 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:56.773176 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:56.870523 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd0d83-858d-4e5d-aa43-1e79014f3250 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:56.902573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022519e6-5a6b-4692-9f1f-a79f8a29f950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:56.903169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022519e6-5a6b-4692-9f1f-a79f8a29f950 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:56.903515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022519e6-5a6b-4692-9f1f-a79f8a29f950 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:56.903712 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022519e6-5a6b-4692-9f1f-a79f8a29f950 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:56.904370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022519e6-5a6b-4692-9f1f-a79f8a29f950 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:56.904665 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022519e6-5a6b-4692-9f1f-a79f8a29f950 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:56.906012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022519e6-5a6b-4692-9f1f-a79f8a29f950 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:56.910928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 508/1019] 10.222.0.27 () {56 vars in 1112 bytes} [Thu Sep 8 15:55:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:56.916734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cccd4d-466a-41ec-9c4a-7bb779d025f2 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:57.018128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:57.021987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cccd4d-466a-41ec-9c4a-7bb779d025f2 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:57.057639 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a2293a-c4c9-4ad8-ad4c-485f14f927e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:57.058243 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2293a-c4c9-4ad8-ad4c-485f14f927e2 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:57.058518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2293a-c4c9-4ad8-ad4c-485f14f927e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:57.058780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a2293a-c4c9-4ad8-ad4c-485f14f927e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:57.059467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2293a-c4c9-4ad8-ad4c-485f14f927e2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:57.060841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a2293a-c4c9-4ad8-ad4c-485f14f927e2 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:57.081672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 512/1020] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 15:55:56 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 15:55:57.613999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b234ec-61b2-48d9-8302-c16428e8ccfd None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:57.614556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b234ec-61b2-48d9-8302-c16428e8ccfd None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:57.614556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b234ec-61b2-48d9-8302-c16428e8ccfd None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:57.616125 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 509/1021] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:57.621272 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7d925-05c4-43e5-87c6-78eef54aa215 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:57.621635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7d925-05c4-43e5-87c6-78eef54aa215 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:57.621970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb7d925-05c4-43e5-87c6-78eef54aa215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:57.640978 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-ffb7d925-05c4-43e5-87c6-78eef54aa215 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:57.642167 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ffb7d925-05c4-43e5-87c6-78eef54aa215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:57.719480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 513/1022] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:57.983930 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b234ec-61b2-48d9-8302-c16428e8ccfd None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:58.016228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fa185-2bf7-4328-b5d0-5203dfeda606 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:58.017137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fa185-2bf7-4328-b5d0-5203dfeda606 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:58.017408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4fa185-2bf7-4328-b5d0-5203dfeda606 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:58.019027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 514/1023] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:55:58.024116 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982627b-0fce-4be1-906e-57625c8f9210 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:58.024399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982627b-0fce-4be1-906e-57625c8f9210 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:58.024667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982627b-0fce-4be1-906e-57625c8f9210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:58.045010 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-3982627b-0fce-4be1-906e-57625c8f9210 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:58.046233 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3982627b-0fce-4be1-906e-57625c8f9210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:58.056180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:58.095882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b234ec-61b2-48d9-8302-c16428e8ccfd None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:58.127745 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 515/1024] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 15:55:58.128951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d76408-3958-473b-b49f-692db75c91a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:58.129586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d76408-3958-473b-b49f-692db75c91a9 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:58.129857 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d76408-3958-473b-b49f-692db75c91a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:58.130125 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d76408-3958-473b-b49f-692db75c91a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:58.131224 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d76408-3958-473b-b49f-692db75c91a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:58.132677 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d76408-3958-473b-b49f-692db75c91a9 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:58.135479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 510/1025] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:58.139926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982627b-0fce-4be1-906e-57625c8f9210 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:58.154720 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:58.245234 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982627b-0fce-4be1-906e-57625c8f9210 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:58.279798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4815d-87be-491d-a3fc-7e90b7265258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:58.280408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4815d-87be-491d-a3fc-7e90b7265258 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:58.280678 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4815d-87be-491d-a3fc-7e90b7265258 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:58.280954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4815d-87be-491d-a3fc-7e90b7265258 None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:58.281606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4815d-87be-491d-a3fc-7e90b7265258 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:58.283151 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4815d-87be-491d-a3fc-7e90b7265258 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:58.306361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 516/1026] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:55:58 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 15:55:58.383042 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d76408-3958-473b-b49f-692db75c91a9 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:58.492233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d76408-3958-473b-b49f-692db75c91a9 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:58.526655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f504c1-3480-4f51-be3c-33f87894f608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:58.527301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f504c1-3480-4f51-be3c-33f87894f608 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:58.527653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f504c1-3480-4f51-be3c-33f87894f608 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:58.527924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f504c1-3480-4f51-be3c-33f87894f608 None admin] PATH_INFO: `/v3/services/image` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:58.528569 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f504c1-3480-4f51-be3c-33f87894f608 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:58.529914 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f504c1-3480-4f51-be3c-33f87894f608 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:58.531856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-a1f504c1-3480-4f51-be3c-33f87894f608 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:58.539776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:58.540806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:58.541676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:58.541676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:58.541676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:58.541676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:58.541676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:58.541676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:58.541676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:58.541676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:58.541676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:58.541676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:58.541676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:55:58.541676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:55:58.541676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 15:55:58.541676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:55:58.541676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 511/1027] 10.222.0.27 () {56 vars in 1105 bytes} [Thu Sep 8 15:55:58 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:58.547208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4815d-87be-491d-a3fc-7e90b7265258 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:58.624135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:58.660278 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4815d-87be-491d-a3fc-7e90b7265258 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:58.694260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68463447-495d-457d-be83-bff4648c56e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:58.694879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68463447-495d-457d-be83-bff4648c56e7 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:58.695150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68463447-495d-457d-be83-bff4648c56e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:58.695417 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68463447-495d-457d-be83-bff4648c56e7 None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:58.696103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68463447-495d-457d-be83-bff4648c56e7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:58.696387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68463447-495d-457d-be83-bff4648c56e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:58.697838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68463447-495d-457d-be83-bff4648c56e7 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:58.703105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 517/1028] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:58.708775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f504c1-3480-4f51-be3c-33f87894f608 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:58.813340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f504c1-3480-4f51-be3c-33f87894f608 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:58.850557 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76ded3-ec8f-4cdd-823a-5073ece03c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:58.851177 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76ded3-ec8f-4cdd-823a-5073ece03c60 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:58.851421 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76ded3-ec8f-4cdd-823a-5073ece03c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:58.851756 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae76ded3-ec8f-4cdd-823a-5073ece03c60 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:58.852357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76ded3-ec8f-4cdd-823a-5073ece03c60 None admin] RBAC: Adding query filter params (type=image) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:58.852687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76ded3-ec8f-4cdd-823a-5073ece03c60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:58.854065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae76ded3-ec8f-4cdd-823a-5073ece03c60 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:58.863537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 512/1029] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:55:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:58.866036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68463447-495d-457d-be83-bff4648c56e7 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:58.971068 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68463447-495d-457d-be83-bff4648c56e7 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:59.013346 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:59.022259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084b7a1-96d4-4cb8-a865-3258ee3f6ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:59.023152 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084b7a1-96d4-4cb8-a865-3258ee3f6ecb None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:59.023570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084b7a1-96d4-4cb8-a865-3258ee3f6ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:59.023943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084b7a1-96d4-4cb8-a865-3258ee3f6ecb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:59.025035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084b7a1-96d4-4cb8-a865-3258ee3f6ecb None admin] RBAC: Adding query filter params (interface=public, service_id=3db77a5ff53d428d8a4163309a8dc696, region_id=RegionOne) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:55:59.025503 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084b7a1-96d4-4cb8-a865-3258ee3f6ecb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:59.030084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084b7a1-96d4-4cb8-a865-3258ee3f6ecb None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:59.039633 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 518/1030] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Sep 8 15:55:58 2022] GET /identity/v3/endpoints?service_id=3db77a5ff53d428d8a4163309a8dc696&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:55:59.041345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76ded3-ec8f-4cdd-823a-5073ece03c60 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:59.114238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:59.155177 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae76ded3-ec8f-4cdd-823a-5073ece03c60 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:59.198457 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b7078-dd6c-44e3-b269-2011628b5d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:59.199108 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b7078-dd6c-44e3-b269-2011628b5d63 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:59.199383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b7078-dd6c-44e3-b269-2011628b5d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:59.199686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b7078-dd6c-44e3-b269-2011628b5d63 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:59.200325 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b7078-dd6c-44e3-b269-2011628b5d63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:55:59.201745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b7078-dd6c-44e3-b269-2011628b5d63 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:55:59.210284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 513/1031] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Sep 8 15:55:59 2022] GET /identity/v3/services => generated 1957 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 15:55:59.329055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af728a4-f271-4427-a9b0-ef8f43a98fd6 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:59.329399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af728a4-f271-4427-a9b0-ef8f43a98fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:59.329664 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af728a4-f271-4427-a9b0-ef8f43a98fd6 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:59.330982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 519/1032] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:55:59.337601 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61021533-13f9-4ee0-af4b-349d67d7f499 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:59.337866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61021533-13f9-4ee0-af4b-349d67d7f499 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:59.338234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61021533-13f9-4ee0-af4b-349d67d7f499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:59.359077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-61021533-13f9-4ee0-af4b-349d67d7f499 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:55:59.360355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-61021533-13f9-4ee0-af4b-349d67d7f499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:55:59.437036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 514/1033] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:55:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:55:59.701374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af728a4-f271-4427-a9b0-ef8f43a98fd6 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:59.810357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af728a4-f271-4427-a9b0-ef8f43a98fd6 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:55:59.845517 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d0c8a-da6e-427f-8a7a-c9b0e3c34c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:55:59.846152 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d0c8a-da6e-427f-8a7a-c9b0e3c34c14 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:55:59.846427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d0c8a-da6e-427f-8a7a-c9b0e3c34c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:55:59.846695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d0c8a-da6e-427f-8a7a-c9b0e3c34c14 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:55:59.857561 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d0c8a-da6e-427f-8a7a-c9b0e3c34c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-088d0c8a-da6e-427f-8a7a-c9b0e3c34c14 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:55:59.862585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:55:59.863564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:55:59.868563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:55:59.868563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:55:59.868563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:55:59.868563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:55:59.868563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:55:59.868563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:55:59.868563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:55:59.868563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:55:59.868563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:55:59.868563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:55:59.868563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 15:55:59.868563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:55:59.868563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 520/1034] 10.222.0.27 () {56 vars in 1120 bytes} [Thu Sep 8 15:55:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:55:59.870334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61021533-13f9-4ee0-af4b-349d67d7f499 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:55:59.928439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:55:59.976849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61021533-13f9-4ee0-af4b-349d67d7f499 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:00.011889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5d4bc-b24a-4453-8e95-8cd02e607863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:00.012500 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5d4bc-b24a-4453-8e95-8cd02e607863 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:00.012769 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5d4bc-b24a-4453-8e95-8cd02e607863 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:00.013036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5d4bc-b24a-4453-8e95-8cd02e607863 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:00.013701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5d4bc-b24a-4453-8e95-8cd02e607863 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:00.013999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5d4bc-b24a-4453-8e95-8cd02e607863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:00.015359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5d4bc-b24a-4453-8e95-8cd02e607863 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:00.023677 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 515/1035] 10.222.0.27 () {56 vars in 1115 bytes} [Thu Sep 8 15:55:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:00.029384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d0c8a-da6e-427f-8a7a-c9b0e3c34c14 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:00.087512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:00.136201 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d0c8a-da6e-427f-8a7a-c9b0e3c34c14 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:00.175982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7451a6d-f48c-43ce-a15b-a90306547b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:00.176611 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7451a6d-f48c-43ce-a15b-a90306547b89 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:00.176884 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7451a6d-f48c-43ce-a15b-a90306547b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:00.177148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7451a6d-f48c-43ce-a15b-a90306547b89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:00.193211 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7451a6d-f48c-43ce-a15b-a90306547b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:00.194663 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7451a6d-f48c-43ce-a15b-a90306547b89 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-d7451a6d-f48c-43ce-a15b-a90306547b89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:56:00.208675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:00.209644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:56:00.210603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:56:00.210603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:56:00.210603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:56:00.214928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 521/1036] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:56:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:00.216243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5d4bc-b24a-4453-8e95-8cd02e607863 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:00.316704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5d4bc-b24a-4453-8e95-8cd02e607863 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:00.333844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:00.352787 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30bb0b-cfb2-4ee7-b469-92c1f84341fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:00.353405 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30bb0b-cfb2-4ee7-b469-92c1f84341fa None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:00.353678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30bb0b-cfb2-4ee7-b469-92c1f84341fa None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:00.353940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30bb0b-cfb2-4ee7-b469-92c1f84341fa None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:00.354577 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30bb0b-cfb2-4ee7-b469-92c1f84341fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:00.354869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30bb0b-cfb2-4ee7-b469-92c1f84341fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:00.356255 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30bb0b-cfb2-4ee7-b469-92c1f84341fa None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:00.373071 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 516/1037] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:00.374255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7451a6d-f48c-43ce-a15b-a90306547b89 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:00.481686 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7451a6d-f48c-43ce-a15b-a90306547b89 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:00.491161 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:00.513155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac5d7d-12be-4298-8a8c-64c4c4e980d0 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:00.513437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac5d7d-12be-4298-8a8c-64c4c4e980d0 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:00.513703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac5d7d-12be-4298-8a8c-64c4c4e980d0 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:00.515223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 517/1038] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:56:00.518072 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4910a3-8030-46c5-acfc-7b321c56dbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:00.518685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4910a3-8030-46c5-acfc-7b321c56dbad None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:00.518966 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4910a3-8030-46c5-acfc-7b321c56dbad None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:00.519229 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4910a3-8030-46c5-acfc-7b321c56dbad None admin] PATH_INFO: `/v3/users/nova` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:00.520283 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e744cd-ae19-4175-ab66-3cff87c2a19f None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:00.520565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e744cd-ae19-4175-ab66-3cff87c2a19f None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:00.520842 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e744cd-ae19-4175-ab66-3cff87c2a19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:00.527149 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4910a3-8030-46c5-acfc-7b321c56dbad None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:00.528608 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4910a3-8030-46c5-acfc-7b321c56dbad None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:00.538732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-f0e744cd-ae19-4175-ab66-3cff87c2a19f None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-2f4910a3-8030-46c5-acfc-7b321c56dbad None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:56:00.539841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:00.540781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:00.542395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:00.542395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:00.542395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:56:00.542395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:56:00.542395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:56:00.542395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:56:00.542395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:56:00.542395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:56:00.542395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:56:00.542395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:56:00.543092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f0e744cd-ae19-4175-ab66-3cff87c2a19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:00.543178 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 522/1039] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Sep 8 15:56:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:56:00.546443 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4910a3-8030-46c5-acfc-7b321c56dbad None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:00.612461 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 518/1040] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:00.653646 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4910a3-8030-46c5-acfc-7b321c56dbad None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:00.688206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9f9af-ed4e-4a3a-9981-39f4d0d727ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:00.688863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9f9af-ed4e-4a3a-9981-39f4d0d727ac None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:00.689140 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9f9af-ed4e-4a3a-9981-39f4d0d727ac None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:00.689404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9f9af-ed4e-4a3a-9981-39f4d0d727ac None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:00.690136 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9f9af-ed4e-4a3a-9981-39f4d0d727ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:00.690439 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9f9af-ed4e-4a3a-9981-39f4d0d727ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:00.691911 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9f9af-ed4e-4a3a-9981-39f4d0d727ac None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:00.709259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 523/1041] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Sep 8 15:56:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:56:00.715280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e744cd-ae19-4175-ab66-3cff87c2a19f None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:00.813675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e744cd-ae19-4175-ab66-3cff87c2a19f None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:00.822926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:00.848935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f539781-4fe4-4bc8-9202-789fe49bcbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:00.849555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f539781-4fe4-4bc8-9202-789fe49bcbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:00.849834 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f539781-4fe4-4bc8-9202-789fe49bcbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:00.850101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f539781-4fe4-4bc8-9202-789fe49bcbc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:00.879774 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9f9af-ed4e-4a3a-9981-39f4d0d727ac None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:00.907368 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:00.920891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f539781-4fe4-4bc8-9202-789fe49bcbc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:00.922359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f539781-4fe4-4bc8-9202-789fe49bcbc2 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:00.986469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9f9af-ed4e-4a3a-9981-39f4d0d727ac None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:01.023274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe43727-5055-46ed-9b00-2cb042c6f805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:01.023930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe43727-5055-46ed-9b00-2cb042c6f805 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:01.024211 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe43727-5055-46ed-9b00-2cb042c6f805 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:01.024478 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe43727-5055-46ed-9b00-2cb042c6f805 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:01.031170 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 519/1042] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:56:00 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:56:01.038698 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f539781-4fe4-4bc8-9202-789fe49bcbc2 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:01.043855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe43727-5055-46ed-9b00-2cb042c6f805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-9fe43727-5055-46ed-9b00-2cb042c6f805 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:56:01.048991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:01.049943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:01.050806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:01.050806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:01.050806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:01.050806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:01.050806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:56:01.050806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:56:01.050806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:56:01.050806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:56:01.050806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:56:01.050806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:56:01.050806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:01.050806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:56:01.050806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 524/1043] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:56:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:56:01.051973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:01.056837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe43727-5055-46ed-9b00-2cb042c6f805 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:01.150066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:01.152453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f539781-4fe4-4bc8-9202-789fe49bcbc2 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:01.168629 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe43727-5055-46ed-9b00-2cb042c6f805 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:01.189864 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5a692-b748-4a8e-a976-c2ca4fce82db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:01.190511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5a692-b748-4a8e-a976-c2ca4fce82db None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:01.190793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5a692-b748-4a8e-a976-c2ca4fce82db None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:01.191100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5a692-b748-4a8e-a976-c2ca4fce82db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:01.202351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5a692-b748-4a8e-a976-c2ca4fce82db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:01.204354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5a692-b748-4a8e-a976-c2ca4fce82db None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:01.208478 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36649848-19d1-4705-bf1d-25fe0775c9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:01.209596 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36649848-19d1-4705-bf1d-25fe0775c9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:01.209876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36649848-19d1-4705-bf1d-25fe0775c9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:01.210148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36649848-19d1-4705-bf1d-25fe0775c9c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:01.211190 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36649848-19d1-4705-bf1d-25fe0775c9c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:01.211521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36649848-19d1-4705-bf1d-25fe0775c9c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:01.212896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36649848-19d1-4705-bf1d-25fe0775c9c4 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-f2e5a692-b748-4a8e-a976-c2ca4fce82db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:56:01.220715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:01.221647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:56:01.222554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:56:01.222554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:56:01.222554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:01.222727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 525/1044] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:56:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:01.228603 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 520/1045] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:56:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:01.229338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5a692-b748-4a8e-a976-c2ca4fce82db None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:01.229441 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36649848-19d1-4705-bf1d-25fe0775c9c4 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:01.297397 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:01.347057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36649848-19d1-4705-bf1d-25fe0775c9c4 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:01.348266 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5a692-b748-4a8e-a976-c2ca4fce82db None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:01.383302 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18fe9cd-8210-4f15-88e7-e9ca3beb0d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:01.383950 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fe9cd-8210-4f15-88e7-e9ca3beb0d83 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:01.384225 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fe9cd-8210-4f15-88e7-e9ca3beb0d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:01.384501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18fe9cd-8210-4f15-88e7-e9ca3beb0d83 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:01.385145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fe9cd-8210-4f15-88e7-e9ca3beb0d83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:01.385454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fe9cd-8210-4f15-88e7-e9ca3beb0d83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:01.386831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18fe9cd-8210-4f15-88e7-e9ca3beb0d83 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:01.388758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cefa52-6ca6-4929-9dc3-82441533ec91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:01.389409 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cefa52-6ca6-4929-9dc3-82441533ec91 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:01.389686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cefa52-6ca6-4929-9dc3-82441533ec91 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:01.389956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cefa52-6ca6-4929-9dc3-82441533ec91 None admin] PATH_INFO: `/v3/users/4e50afe7c25d40c5b12ad33ae8a4fa26` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:01.398700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 526/1046] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:56:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:56:01.399980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20929184-0561-4ef7-8700-1002a89bfe5a None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:01.400290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20929184-0561-4ef7-8700-1002a89bfe5a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:01.400501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20929184-0561-4ef7-8700-1002a89bfe5a None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:01.403416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 527/1047] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:01.404722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20929184-0561-4ef7-8700-1002a89bfe5a None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:01.407045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cefa52-6ca6-4929-9dc3-82441533ec91 None admin] RBAC: Authorizing `identity:get_user(user_id=4e50afe7c25d40c5b12ad33ae8a4fa26)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:01.408686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cefa52-6ca6-4929-9dc3-82441533ec91 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:01.410398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 521/1048] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:56:01 2022] GET /identity/v3/users/4e50afe7c25d40c5b12ad33ae8a4fa26 => generated 285 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:01.411615 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061764a-f4d7-4322-975b-5bd8cd35ec78 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:01.411959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061764a-f4d7-4322-975b-5bd8cd35ec78 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:01.412231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061764a-f4d7-4322-975b-5bd8cd35ec78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:01.419307 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:01.430330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-d061764a-f4d7-4322-975b-5bd8cd35ec78 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:01.431560 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d061764a-f4d7-4322-975b-5bd8cd35ec78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:01.496879 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 522/1049] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:01.497708 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061764a-f4d7-4322-975b-5bd8cd35ec78 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:01.512147 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20929184-0561-4ef7-8700-1002a89bfe5a None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:01.547466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:01.550452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468b492-e2cd-4ecd-bf98-5d91b8dbda7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:01.551146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468b492-e2cd-4ecd-bf98-5d91b8dbda7f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:01.551751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468b492-e2cd-4ecd-bf98-5d91b8dbda7f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:01.551751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4468b492-e2cd-4ecd-bf98-5d91b8dbda7f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:01.559572 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468b492-e2cd-4ecd-bf98-5d91b8dbda7f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:01.561023 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4468b492-e2cd-4ecd-bf98-5d91b8dbda7f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:01.572661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-4468b492-e2cd-4ecd-bf98-5d91b8dbda7f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:56:01.572661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:01.572661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:01.572661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:01.572661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:01.572661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:01.572661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:01.572661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:01.572661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:01.572661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:01.572661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:01.572661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:01.572661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:56:01.572661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:56:01.572661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:01.573598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:01.574537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:56:01.574537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:56:01.574537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:56:01.574537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:56:01.574537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:56:01.574537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:56:01.574537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:56:01.574537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:56:01.574537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 528/1050] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:56:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:01.579208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468b492-e2cd-4ecd-bf98-5d91b8dbda7f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:01.605164 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061764a-f4d7-4322-975b-5bd8cd35ec78 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:01.641820 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed1fe8-8e7a-490a-b1d8-d148fa11dd76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:01.642429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed1fe8-8e7a-490a-b1d8-d148fa11dd76 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:01.642705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed1fe8-8e7a-490a-b1d8-d148fa11dd76 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:01.642975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed1fe8-8e7a-490a-b1d8-d148fa11dd76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:01.649055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:01.686239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4468b492-e2cd-4ecd-bf98-5d91b8dbda7f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:01.715560 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed1fe8-8e7a-490a-b1d8-d148fa11dd76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:01.717020 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ed1fe8-8e7a-490a-b1d8-d148fa11dd76 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:01.727038 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fba88-3619-4aa1-81fd-d9a347a89a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:01.727705 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fba88-3619-4aa1-81fd-d9a347a89a51 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:01.727985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fba88-3619-4aa1-81fd-d9a347a89a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:01.728258 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fba88-3619-4aa1-81fd-d9a347a89a51 None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:01.729332 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fba88-3619-4aa1-81fd-d9a347a89a51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:01.729644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fba88-3619-4aa1-81fd-d9a347a89a51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:01.731411 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6fba88-3619-4aa1-81fd-d9a347a89a51 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:01.745903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 529/1051] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:56:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:56:01.752128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fba88-3619-4aa1-81fd-d9a347a89a51 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:01.771130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:01.823098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 523/1052] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:56:01 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:56:01.824235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed1fe8-8e7a-490a-b1d8-d148fa11dd76 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:01.860217 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fba88-3619-4aa1-81fd-d9a347a89a51 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:01.888508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:01.896939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d20bab-b285-407c-965b-b9688fe37dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:01.897531 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d20bab-b285-407c-965b-b9688fe37dbe None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:01.897819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d20bab-b285-407c-965b-b9688fe37dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:01.898085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d20bab-b285-407c-965b-b9688fe37dbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:01.898808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d20bab-b285-407c-965b-b9688fe37dbe None admin] RBAC: Adding query filter params (role.id=f752d851596f48a3bb97ad5d8f09d763, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=6e9209e77ce44b128e25a8ddcf78f8b5) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:01.899107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d20bab-b285-407c-965b-b9688fe37dbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:01.900564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d20bab-b285-407c-965b-b9688fe37dbe None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:01.906150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 530/1053] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:56:01 2022] GET /identity/v3/role_assignments?user.id=6e9209e77ce44b128e25a8ddcf78f8b5&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=f752d851596f48a3bb97ad5d8f09d763 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:56:01.906905 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d20bab-b285-407c-965b-b9688fe37dbe None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:01.925236 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ed1fe8-8e7a-490a-b1d8-d148fa11dd76 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:01.965778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27ffd7-8930-49e2-939d-3b42c812fd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:01.966351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27ffd7-8930-49e2-939d-3b42c812fd63 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:01.966629 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27ffd7-8930-49e2-939d-3b42c812fd63 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:01.966900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb27ffd7-8930-49e2-939d-3b42c812fd63 None admin] PATH_INFO: `/v3/services/image` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:01.967674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27ffd7-8930-49e2-939d-3b42c812fd63 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:01.969220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb27ffd7-8930-49e2-939d-3b42c812fd63 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-cb27ffd7-8930-49e2-939d-3b42c812fd63 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:01.980740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:01.982051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:01.983247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:01.983247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:01.983247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:01.983247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:01.983247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:01.983247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:01.983247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:01.983247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:01.983247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:01.983247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:01.983247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:56:01.983247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:56:01.983247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 15:56:01.983247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:01.983247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 524/1054] 10.222.0.27 () {56 vars in 1105 bytes} [Thu Sep 8 15:56:01 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:01.988220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27ffd7-8930-49e2-939d-3b42c812fd63 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:02.005323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:02.022178 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d20bab-b285-407c-965b-b9688fe37dbe None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:02.061548 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f58ad8-ff48-4eb3-925a-76d6de43d3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:02.062251 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f58ad8-ff48-4eb3-925a-76d6de43d3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:02.062454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f58ad8-ff48-4eb3-925a-76d6de43d3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:02.062722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f58ad8-ff48-4eb3-925a-76d6de43d3c6 None admin] PATH_INFO: `/v3/projects/037ba1f2c61344058dba200c677d9a37` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:02.064091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f58ad8-ff48-4eb3-925a-76d6de43d3c6 None admin] RBAC: Authorizing `identity:get_project(project_id=037ba1f2c61344058dba200c677d9a37)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:02.065551 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f58ad8-ff48-4eb3-925a-76d6de43d3c6 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:02.067263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 531/1055] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:56:01 2022] GET /identity/v3/projects/037ba1f2c61344058dba200c677d9a37 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:02.073998 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f58ad8-ff48-4eb3-925a-76d6de43d3c6 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:02.121786 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb27ffd7-8930-49e2-939d-3b42c812fd63 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:02.138178 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:02.159215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b47331-cc6f-4c66-9206-4fdb91fa0593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:02.159858 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b47331-cc6f-4c66-9206-4fdb91fa0593 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:02.160129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b47331-cc6f-4c66-9206-4fdb91fa0593 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:02.160396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b47331-cc6f-4c66-9206-4fdb91fa0593 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:02.161029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b47331-cc6f-4c66-9206-4fdb91fa0593 None admin] RBAC: Adding query filter params (name=image) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:02.161321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b47331-cc6f-4c66-9206-4fdb91fa0593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:02.162702 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b47331-cc6f-4c66-9206-4fdb91fa0593 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:02.167894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 525/1056] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:56:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:56:02.173467 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b47331-cc6f-4c66-9206-4fdb91fa0593 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:02.190785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f58ad8-ff48-4eb3-925a-76d6de43d3c6 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:02.233593 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83dd67-b9b7-4e39-93a1-22c97a208560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:02.234310 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83dd67-b9b7-4e39-93a1-22c97a208560 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:02.234593 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83dd67-b9b7-4e39-93a1-22c97a208560 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:02.235166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e83dd67-b9b7-4e39-93a1-22c97a208560 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:02.236247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83dd67-b9b7-4e39-93a1-22c97a208560 None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=037ba1f2c61344058dba200c677d9a37, user.id=4e50afe7c25d40c5b12ad33ae8a4fa26) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:02.236559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83dd67-b9b7-4e39-93a1-22c97a208560 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:02.238731 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e83dd67-b9b7-4e39-93a1-22c97a208560 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:02.244616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 532/1057] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:56:02 2022] GET /identity/v3/role_assignments?user.id=4e50afe7c25d40c5b12ad33ae8a4fa26&scope.project.id=037ba1f2c61344058dba200c677d9a37&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:02.266822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:02.295131 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b47331-cc6f-4c66-9206-4fdb91fa0593 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:02.333670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f681b0a2-3fb6-4999-b1a6-bb5562639fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:02.334469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681b0a2-3fb6-4999-b1a6-bb5562639fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:02.334895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681b0a2-3fb6-4999-b1a6-bb5562639fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:02.335266 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f681b0a2-3fb6-4999-b1a6-bb5562639fc4 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:02.336404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681b0a2-3fb6-4999-b1a6-bb5562639fc4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:02.336793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681b0a2-3fb6-4999-b1a6-bb5562639fc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:02.338286 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f681b0a2-3fb6-4999-b1a6-bb5562639fc4 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:02.349958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 526/1058] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:56:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:56:02.351364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83dd67-b9b7-4e39-93a1-22c97a208560 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:02.462866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e83dd67-b9b7-4e39-93a1-22c97a208560 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:02.487016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:02.498918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85cbc15-688e-429e-8954-07aff010a6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:02.499550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cbc15-688e-429e-8954-07aff010a6b6 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:02.499826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cbc15-688e-429e-8954-07aff010a6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:02.500092 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85cbc15-688e-429e-8954-07aff010a6b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:02.500764 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cbc15-688e-429e-8954-07aff010a6b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:02.502182 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85cbc15-688e-429e-8954-07aff010a6b6 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:02.523882 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 533/1059] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 15:56:02 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:56:04.022143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758fcd4-c043-4f15-8172-ff70994a96b9 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:04.022701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758fcd4-c043-4f15-8172-ff70994a96b9 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:04.022701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758fcd4-c043-4f15-8172-ff70994a96b9 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:04.024258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 527/1060] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:04.029170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aa9a4-090e-4fb2-a2f5-d754a83bc26f None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:04.029605 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aa9a4-090e-4fb2-a2f5-d754a83bc26f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:04.029750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681aa9a4-090e-4fb2-a2f5-d754a83bc26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:04.048958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-681aa9a4-090e-4fb2-a2f5-d754a83bc26f None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:04.050159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-681aa9a4-090e-4fb2-a2f5-d754a83bc26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:04.135324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 534/1061] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:04.372655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df10839-bb42-4060-996f-9135fbe8f87f None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:04.373052 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df10839-bb42-4060-996f-9135fbe8f87f None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:04.373327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df10839-bb42-4060-996f-9135fbe8f87f None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:04.374747 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 528/1062] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:04.380162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549bfa5-0938-4674-aefb-da01bf9275c1 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:04.380448 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549bfa5-0938-4674-aefb-da01bf9275c1 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:04.380715 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549bfa5-0938-4674-aefb-da01bf9275c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:04.392676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df10839-bb42-4060-996f-9135fbe8f87f None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:04.399605 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f549bfa5-0938-4674-aefb-da01bf9275c1 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:04.400756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f549bfa5-0938-4674-aefb-da01bf9275c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:04.470762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 535/1063] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:04.494295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df10839-bb42-4060-996f-9135fbe8f87f None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:04.528594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307eb38b-2020-4881-b9cb-47e30082159f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:04.529224 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307eb38b-2020-4881-b9cb-47e30082159f None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:04.529471 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307eb38b-2020-4881-b9cb-47e30082159f None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:04.529754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307eb38b-2020-4881-b9cb-47e30082159f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:04.537531 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:04.541493 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307eb38b-2020-4881-b9cb-47e30082159f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-307eb38b-2020-4881-b9cb-47e30082159f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:56:04.546563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:04.547753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:04.548952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:04.548952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:04.548952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:04.548952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:04.548952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:56:04.548952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:56:04.548952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:56:04.548952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:56:04.548952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:56:04.548952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:56:04.548952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 15:56:04.548952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:04.552948 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 529/1064] 10.222.0.27 () {56 vars in 1120 bytes} [Thu Sep 8 15:56:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:56:04.553818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549bfa5-0938-4674-aefb-da01bf9275c1 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:04.599931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:04.666431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549bfa5-0938-4674-aefb-da01bf9275c1 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:04.687398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9658760-75b9-4004-a3d5-23f2754a11e8 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:04.687707 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9658760-75b9-4004-a3d5-23f2754a11e8 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:04.687985 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9658760-75b9-4004-a3d5-23f2754a11e8 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:04.689415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 530/1065] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:56:04.694234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf21277-3194-4a3d-abcd-2c88a69c4947 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:04.694511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf21277-3194-4a3d-abcd-2c88a69c4947 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:04.694780 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf21277-3194-4a3d-abcd-2c88a69c4947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:04.706790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51d71b-cf20-48f2-912c-ff03b35880b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:04.707411 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51d71b-cf20-48f2-912c-ff03b35880b6 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:04.707715 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51d71b-cf20-48f2-912c-ff03b35880b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:04.708087 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51d71b-cf20-48f2-912c-ff03b35880b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:04.708753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51d71b-cf20-48f2-912c-ff03b35880b6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:04.709049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51d71b-cf20-48f2-912c-ff03b35880b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:04.710400 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51d71b-cf20-48f2-912c-ff03b35880b6 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:04.713747 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-ecf21277-3194-4a3d-abcd-2c88a69c4947 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:04.714935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ecf21277-3194-4a3d-abcd-2c88a69c4947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:04.719490 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 536/1066] 10.222.0.27 () {56 vars in 1115 bytes} [Thu Sep 8 15:56:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:04.724475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51d71b-cf20-48f2-912c-ff03b35880b6 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:04.786647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 531/1067] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 15:56:04.787896 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf21277-3194-4a3d-abcd-2c88a69c4947 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:04.822537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:04.826719 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51d71b-cf20-48f2-912c-ff03b35880b6 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:04.861914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15237391-7f09-4598-a536-df28e278c8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:04.862540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15237391-7f09-4598-a536-df28e278c8ac None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:04.862808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15237391-7f09-4598-a536-df28e278c8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:04.863070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15237391-7f09-4598-a536-df28e278c8ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:04.873287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15237391-7f09-4598-a536-df28e278c8ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:04.874849 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15237391-7f09-4598-a536-df28e278c8ac None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-15237391-7f09-4598-a536-df28e278c8ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:56:04.889657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:04.890602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:56:04.892245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:56:04.892245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:56:04.892245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:56:04.892245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 537/1068] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:56:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:56:04.892501 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf21277-3194-4a3d-abcd-2c88a69c4947 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:04.896463 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15237391-7f09-4598-a536-df28e278c8ac None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:04.922123 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:04.929650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1038edb-6cb2-4950-9bf9-c9b9cc553e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:04.930402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1038edb-6cb2-4950-9bf9-c9b9cc553e7a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:04.930683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1038edb-6cb2-4950-9bf9-c9b9cc553e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:04.930954 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1038edb-6cb2-4950-9bf9-c9b9cc553e7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:04.942007 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1038edb-6cb2-4950-9bf9-c9b9cc553e7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-b1038edb-6cb2-4950-9bf9-c9b9cc553e7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:56:04.947070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:04.948130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:04.949107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:04.949107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:04.949107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:04.949107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:04.949107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:56:04.949107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:56:04.949107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:56:04.949107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:56:04.949107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:56:04.949107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:56:04.949107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:04.949107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:04.949107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 532/1069] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:56:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:04.954516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1038edb-6cb2-4950-9bf9-c9b9cc553e7a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:04.995964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15237391-7f09-4598-a536-df28e278c8ac None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:05.032330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970bc51-2e8d-4a68-8433-2ccacd438c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:05.032962 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970bc51-2e8d-4a68-8433-2ccacd438c96 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:05.033203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970bc51-2e8d-4a68-8433-2ccacd438c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:05.033462 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970bc51-2e8d-4a68-8433-2ccacd438c96 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:05.034101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970bc51-2e8d-4a68-8433-2ccacd438c96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:05.034395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970bc51-2e8d-4a68-8433-2ccacd438c96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:05.035798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970bc51-2e8d-4a68-8433-2ccacd438c96 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:05.048176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 538/1070] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:56:05.049845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970bc51-2e8d-4a68-8433-2ccacd438c96 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:05.061260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1038edb-6cb2-4950-9bf9-c9b9cc553e7a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:05.062104 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:05.097492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248aa479-43ee-4920-86c0-c7f83ea240cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:05.098124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248aa479-43ee-4920-86c0-c7f83ea240cf None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:05.098430 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248aa479-43ee-4920-86c0-c7f83ea240cf None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:05.098680 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248aa479-43ee-4920-86c0-c7f83ea240cf None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:05.099347 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248aa479-43ee-4920-86c0-c7f83ea240cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:05.099673 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248aa479-43ee-4920-86c0-c7f83ea240cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:05.101028 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248aa479-43ee-4920-86c0-c7f83ea240cf None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:05.109491 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 533/1071] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:56:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:05.110307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248aa479-43ee-4920-86c0-c7f83ea240cf None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:05.148406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970bc51-2e8d-4a68-8433-2ccacd438c96 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:05.166595 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:05.182960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5267a-52d6-4e3b-89da-2a3d08656430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:05.183529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5267a-52d6-4e3b-89da-2a3d08656430 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:05.183809 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5267a-52d6-4e3b-89da-2a3d08656430 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:05.184532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5267a-52d6-4e3b-89da-2a3d08656430 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:05.215358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248aa479-43ee-4920-86c0-c7f83ea240cf None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:05.249572 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9182ac-88f8-49dc-9828-c887ae9411ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:05.250220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9182ac-88f8-49dc-9828-c887ae9411ed None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:05.250503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9182ac-88f8-49dc-9828-c887ae9411ed None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:05.250775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9182ac-88f8-49dc-9828-c887ae9411ed None admin] PATH_INFO: `/v3/users/nova` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:05.251422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5267a-52d6-4e3b-89da-2a3d08656430 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:05.252906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5267a-52d6-4e3b-89da-2a3d08656430 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:05.259157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9182ac-88f8-49dc-9828-c887ae9411ed None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:05.260673 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9182ac-88f8-49dc-9828-c887ae9411ed None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:05.264769 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-ac9182ac-88f8-49dc-9828-c887ae9411ed None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:56:05.273000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:05.273958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:05.280303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:05.280303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:05.280303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:56:05.280303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:56:05.280303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:56:05.280303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:56:05.280303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:56:05.280303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:56:05.280303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:56:05.280303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:05.280303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 534/1072] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Sep 8 15:56:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:05.280303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9182ac-88f8-49dc-9828-c887ae9411ed None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:05.354910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 539/1073] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:56:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 15:56:05.355761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5267a-52d6-4e3b-89da-2a3d08656430 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:05.378356 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9182ac-88f8-49dc-9828-c887ae9411ed None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:05.390905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:05.412487 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5de206-5778-47c3-8a52-ae25d1ad61a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:05.413109 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5de206-5778-47c3-8a52-ae25d1ad61a1 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:05.413384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5de206-5778-47c3-8a52-ae25d1ad61a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:05.413655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5de206-5778-47c3-8a52-ae25d1ad61a1 None admin] PATH_INFO: `/v3/users/4e50afe7c25d40c5b12ad33ae8a4fa26` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:05.414779 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5de206-5778-47c3-8a52-ae25d1ad61a1 None admin] RBAC: Authorizing `identity:get_user(user_id=4e50afe7c25d40c5b12ad33ae8a4fa26)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:05.416253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5de206-5778-47c3-8a52-ae25d1ad61a1 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:05.417876 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 535/1074] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:56:05 2022] GET /identity/v3/users/4e50afe7c25d40c5b12ad33ae8a4fa26 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:05.418635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5de206-5778-47c3-8a52-ae25d1ad61a1 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:05.455434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5267a-52d6-4e3b-89da-2a3d08656430 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:05.466321 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:05.489800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e795d48-7e20-41ef-8ea0-a931ec9a4db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:05.490402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e795d48-7e20-41ef-8ea0-a931ec9a4db0 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:05.490673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e795d48-7e20-41ef-8ea0-a931ec9a4db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:05.490933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e795d48-7e20-41ef-8ea0-a931ec9a4db0 None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:05.491682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e795d48-7e20-41ef-8ea0-a931ec9a4db0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:05.491979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e795d48-7e20-41ef-8ea0-a931ec9a4db0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:05.493400 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e795d48-7e20-41ef-8ea0-a931ec9a4db0 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:05.510123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 540/1075] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Sep 8 15:56:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:05.510905 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e795d48-7e20-41ef-8ea0-a931ec9a4db0 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:05.518204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5de206-5778-47c3-8a52-ae25d1ad61a1 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:05.551359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d752018-6481-44d4-9974-23e24756ad8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:05.552013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d752018-6481-44d4-9974-23e24756ad8e None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:05.552284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d752018-6481-44d4-9974-23e24756ad8e None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:05.552548 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d752018-6481-44d4-9974-23e24756ad8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:05.563029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d752018-6481-44d4-9974-23e24756ad8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:05.564543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d752018-6481-44d4-9974-23e24756ad8e None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-2d752018-6481-44d4-9974-23e24756ad8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:56:05.578489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:05.579715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:56:05.580676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:56:05.580676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:56:05.580676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:05.580676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 536/1076] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:05.580915 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d752018-6481-44d4-9974-23e24756ad8e None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:05.615216 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e795d48-7e20-41ef-8ea0-a931ec9a4db0 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:05.615936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:05.648127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce35d6-c0ea-4b58-9da6-9c80b6542e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:05.648755 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce35d6-c0ea-4b58-9da6-9c80b6542e21 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:05.649027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce35d6-c0ea-4b58-9da6-9c80b6542e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:05.649291 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce35d6-c0ea-4b58-9da6-9c80b6542e21 None admin] PATH_INFO: `/v3/projects/037ba1f2c61344058dba200c677d9a37` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:05.650382 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce35d6-c0ea-4b58-9da6-9c80b6542e21 None admin] RBAC: Authorizing `identity:get_project(project_id=037ba1f2c61344058dba200c677d9a37)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:05.651855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce35d6-c0ea-4b58-9da6-9c80b6542e21 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:05.653496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 541/1077] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:56:05 2022] GET /identity/v3/projects/037ba1f2c61344058dba200c677d9a37 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:05.654246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce35d6-c0ea-4b58-9da6-9c80b6542e21 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:05.683661 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d752018-6481-44d4-9974-23e24756ad8e None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:05.691577 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:05.717516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdebcc0-5f52-4d4c-a3ce-9d6c1c32f929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:05.718048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdebcc0-5f52-4d4c-a3ce-9d6c1c32f929 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:05.718320 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdebcc0-5f52-4d4c-a3ce-9d6c1c32f929 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:05.718573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdebcc0-5f52-4d4c-a3ce-9d6c1c32f929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:05.729664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdebcc0-5f52-4d4c-a3ce-9d6c1c32f929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:05.731102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdebcc0-5f52-4d4c-a3ce-9d6c1c32f929 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-5bdebcc0-5f52-4d4c-a3ce-9d6c1c32f929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:56:05.745059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:05.746191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:56:05.747195 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:56:05.747195 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:56:05.747195 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:05.747195 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 537/1078] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:05.747195 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdebcc0-5f52-4d4c-a3ce-9d6c1c32f929 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:05.760713 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce35d6-c0ea-4b58-9da6-9c80b6542e21 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:05.793555 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab25ffc-071d-44c7-81f8-c3b3aa3b8f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:05.794155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab25ffc-071d-44c7-81f8-c3b3aa3b8f82 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:05.794426 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab25ffc-071d-44c7-81f8-c3b3aa3b8f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:05.794672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab25ffc-071d-44c7-81f8-c3b3aa3b8f82 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:05.795306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab25ffc-071d-44c7-81f8-c3b3aa3b8f82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:05.795628 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab25ffc-071d-44c7-81f8-c3b3aa3b8f82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:05.796990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab25ffc-071d-44c7-81f8-c3b3aa3b8f82 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:05.808022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 542/1079] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:05.808788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab25ffc-071d-44c7-81f8-c3b3aa3b8f82 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:05.833841 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:05.854739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdebcc0-5f52-4d4c-a3ce-9d6c1c32f929 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:05.894002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7029b6e-2998-48e6-9285-a7eb2953e060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:05.894789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7029b6e-2998-48e6-9285-a7eb2953e060 None admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:05.895133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7029b6e-2998-48e6-9285-a7eb2953e060 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:05.895480 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7029b6e-2998-48e6-9285-a7eb2953e060 None admin] PATH_INFO: `/v3/projects/037ba1f2c61344058dba200c677d9a37/users/4e50afe7c25d40c5b12ad33ae8a4fa26/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:05.898623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7029b6e-2998-48e6-9285-a7eb2953e060 None admin] RBAC: Authorizing `identity:create_grant(project_id=037ba1f2c61344058dba200c677d9a37, user_id=4e50afe7c25d40c5b12ad33ae8a4fa26, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:05.900231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7029b6e-2998-48e6-9285-a7eb2953e060 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:05.916833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 538/1080] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:56:05 2022] PUT /identity/v3/projects/037ba1f2c61344058dba200c677d9a37/users/4e50afe7c25d40c5b12ad33ae8a4fa26/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:56:05.936917 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:05.945441 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab25ffc-071d-44c7-81f8-c3b3aa3b8f82 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:05.988438 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaea4a0-b961-463a-9f7b-a3f4e24a284f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:05.989137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaea4a0-b961-463a-9f7b-a3f4e24a284f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:05.989506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaea4a0-b961-463a-9f7b-a3f4e24a284f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:05.989870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaea4a0-b961-463a-9f7b-a3f4e24a284f None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:05.990689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaea4a0-b961-463a-9f7b-a3f4e24a284f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:05.991088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaea4a0-b961-463a-9f7b-a3f4e24a284f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:05.993191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaea4a0-b961-463a-9f7b-a3f4e24a284f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:06.007178 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 543/1081] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:06.013764 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7029b6e-2998-48e6-9285-a7eb2953e060 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:06.133924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7029b6e-2998-48e6-9285-a7eb2953e060 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:06.168755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3b6f1-07a4-43ca-b547-a84b8ca41320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:06.169479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b6f1-07a4-43ca-b547-a84b8ca41320 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:06.169748 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b6f1-07a4-43ca-b547-a84b8ca41320 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:06.170011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3b6f1-07a4-43ca-b547-a84b8ca41320 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:06.176931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3b6f1-07a4-43ca-b547-a84b8ca41320 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:06.178361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3b6f1-07a4-43ca-b547-a84b8ca41320 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:06.190654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-ede3b6f1-07a4-43ca-b547-a84b8ca41320 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:56:06.190654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:06.190654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:06.190654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:06.190654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:06.190654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:06.190654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:06.190654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:06.190654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:06.190654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:06.190654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:06.190654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:06.190654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:56:06.190654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:56:06.190654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:06.191714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:06.192823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:56:06.192823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:56:06.192823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:56:06.192823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:56:06.192823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:56:06.192823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:56:06.192823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:56:06.192823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:06.192823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 539/1082] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:56:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:06.199458 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaea4a0-b961-463a-9f7b-a3f4e24a284f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:06.300181 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaea4a0-b961-463a-9f7b-a3f4e24a284f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:06.334996 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894dcba8-bf34-4a55-8b54-7e18b8eaefa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:06.336110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dcba8-bf34-4a55-8b54-7e18b8eaefa6 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:06.336582 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dcba8-bf34-4a55-8b54-7e18b8eaefa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:06.336821 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894dcba8-bf34-4a55-8b54-7e18b8eaefa6 None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:06.337549 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894dcba8-bf34-4a55-8b54-7e18b8eaefa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:06.337865 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894dcba8-bf34-4a55-8b54-7e18b8eaefa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:06.339782 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894dcba8-bf34-4a55-8b54-7e18b8eaefa6 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:06.351621 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 544/1083] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:56:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:06.357698 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3b6f1-07a4-43ca-b547-a84b8ca41320 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:06.362383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:06.459351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3b6f1-07a4-43ca-b547-a84b8ca41320 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:06.493624 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463809b-6a92-4643-9e55-662627288a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:06.494287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463809b-6a92-4643-9e55-662627288a5b None admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:06.494554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463809b-6a92-4643-9e55-662627288a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:06.494818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463809b-6a92-4643-9e55-662627288a5b None admin] PATH_INFO: `/v3/projects/a825a0b1238d4382995da440944f453b/users/6e9209e77ce44b128e25a8ddcf78f8b5/roles/f752d851596f48a3bb97ad5d8f09d763` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:06.497093 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463809b-6a92-4643-9e55-662627288a5b None admin] RBAC: Authorizing `identity:create_grant(project_id=a825a0b1238d4382995da440944f453b, user_id=6e9209e77ce44b128e25a8ddcf78f8b5, role_id=f752d851596f48a3bb97ad5d8f09d763)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:06.498516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463809b-6a92-4643-9e55-662627288a5b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:06.512439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 540/1084] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:56:06 2022] PUT /identity/v3/projects/a825a0b1238d4382995da440944f453b/users/6e9209e77ce44b128e25a8ddcf78f8b5/roles/f752d851596f48a3bb97ad5d8f09d763 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:56:08.048958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cd7ae-51d3-42ef-a233-52530826cceb None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:08.049561 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cd7ae-51d3-42ef-a233-52530826cceb None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:08.049561 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0cd7ae-51d3-42ef-a233-52530826cceb None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:08.050838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 545/1085] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:08.056254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8581f9-0297-476e-970a-99a839ccfb16 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:08.056694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8581f9-0297-476e-970a-99a839ccfb16 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:08.056796 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8581f9-0297-476e-970a-99a839ccfb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:08.076627 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-1b8581f9-0297-476e-970a-99a839ccfb16 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:08.077816 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1b8581f9-0297-476e-970a-99a839ccfb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:08.138684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:08.148713 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 541/1086] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:08.414780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cd7ae-51d3-42ef-a233-52530826cceb None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:08.434971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:08.526689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0cd7ae-51d3-42ef-a233-52530826cceb None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:08.561703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35427864-7f99-4f7a-8cee-a6bd844188a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:08.562347 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35427864-7f99-4f7a-8cee-a6bd844188a6 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:08.562637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35427864-7f99-4f7a-8cee-a6bd844188a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:08.562916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35427864-7f99-4f7a-8cee-a6bd844188a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:08.573845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35427864-7f99-4f7a-8cee-a6bd844188a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-35427864-7f99-4f7a-8cee-a6bd844188a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:56:08.578682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:08.579887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:08.580816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:08.580816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:08.580816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:08.580816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:08.580816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:56:08.580816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:56:08.580816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:56:08.580816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:56:08.580816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:56:08.580816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:56:08.580816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:08.580816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:56:08.580816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 546/1087] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:56:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:08.586464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8581f9-0297-476e-970a-99a839ccfb16 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:08.599014 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b6e1b-07aa-43b5-8e98-7973dcf80851 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:08.599291 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b6e1b-07aa-43b5-8e98-7973dcf80851 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:08.599600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b6e1b-07aa-43b5-8e98-7973dcf80851 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:08.600905 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 547/1088] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:56:08.605732 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de7ed3-a243-44f3-9b69-e0118f410c0c None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:08.606042 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de7ed3-a243-44f3-9b69-e0118f410c0c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:08.606308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de7ed3-a243-44f3-9b69-e0118f410c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:08.624271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-66de7ed3-a243-44f3-9b69-e0118f410c0c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:08.625446 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-66de7ed3-a243-44f3-9b69-e0118f410c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:08.687412 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8581f9-0297-476e-970a-99a839ccfb16 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:08.689313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 548/1089] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 15:56:08.722324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d5bb1-4e6a-42c6-9bca-16aa586ecaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:08.722930 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d5bb1-4e6a-42c6-9bca-16aa586ecaa4 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:08.723168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d5bb1-4e6a-42c6-9bca-16aa586ecaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:08.723503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d5bb1-4e6a-42c6-9bca-16aa586ecaa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:08.724146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d5bb1-4e6a-42c6-9bca-16aa586ecaa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:08.724440 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d5bb1-4e6a-42c6-9bca-16aa586ecaa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:08.725801 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d5bb1-4e6a-42c6-9bca-16aa586ecaa4 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:08.733440 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 542/1090] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:56:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:08.739552 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de7ed3-a243-44f3-9b69-e0118f410c0c None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:08.747949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:08.840693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de7ed3-a243-44f3-9b69-e0118f410c0c None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:08.874820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce6514-05a4-4477-94b6-457070c2517a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:08.875472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce6514-05a4-4477-94b6-457070c2517a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:08.875744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce6514-05a4-4477-94b6-457070c2517a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:08.876057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce6514-05a4-4477-94b6-457070c2517a None admin] PATH_INFO: `/v3/users/4e50afe7c25d40c5b12ad33ae8a4fa26` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:08.877216 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce6514-05a4-4477-94b6-457070c2517a None admin] RBAC: Authorizing `identity:get_user(user_id=4e50afe7c25d40c5b12ad33ae8a4fa26)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:08.878650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce6514-05a4-4477-94b6-457070c2517a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:08.880441 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 549/1091] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:56:08 2022] GET /identity/v3/users/4e50afe7c25d40c5b12ad33ae8a4fa26 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:08.886369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d5bb1-4e6a-42c6-9bca-16aa586ecaa4 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:08.942009 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:08.953025 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce6514-05a4-4477-94b6-457070c2517a None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:08.993150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d5bb1-4e6a-42c6-9bca-16aa586ecaa4 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:09.027795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfee79-4da4-41ad-8328-2dacebf39a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:09.028400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfee79-4da4-41ad-8328-2dacebf39a50 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:09.028673 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfee79-4da4-41ad-8328-2dacebf39a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:09.028957 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfee79-4da4-41ad-8328-2dacebf39a50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:09.031715 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:09.056824 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce6514-05a4-4477-94b6-457070c2517a None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:09.089684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd188f6d-f6e7-46be-8561-63269e6a3e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:09.090302 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd188f6d-f6e7-46be-8561-63269e6a3e4c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:09.090574 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd188f6d-f6e7-46be-8561-63269e6a3e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:09.090837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd188f6d-f6e7-46be-8561-63269e6a3e4c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:09.094327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfee79-4da4-41ad-8328-2dacebf39a50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:09.095801 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfee79-4da4-41ad-8328-2dacebf39a50 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:09.100804 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd188f6d-f6e7-46be-8561-63269e6a3e4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-bd188f6d-f6e7-46be-8561-63269e6a3e4c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:56:09.105580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:09.106697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:09.107787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:09.107787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:09.107787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:09.107787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:09.107787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:56:09.107787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:56:09.107787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:56:09.107787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:56:09.107787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:56:09.107787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:56:09.107787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 15:56:09.107787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:56:09.107787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 550/1092] 10.222.0.27 () {56 vars in 1120 bytes} [Thu Sep 8 15:56:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:56:09.112805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd188f6d-f6e7-46be-8561-63269e6a3e4c None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:09.155403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:09.200731 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 543/1093] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:56:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:56:09.206755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfee79-4da4-41ad-8328-2dacebf39a50 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:09.214372 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd188f6d-f6e7-46be-8561-63269e6a3e4c None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:09.251257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e91dd-e0b9-4a26-a0c4-9b72fd3b6424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:09.252104 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e91dd-e0b9-4a26-a0c4-9b72fd3b6424 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:09.252378 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e91dd-e0b9-4a26-a0c4-9b72fd3b6424 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:09.252644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e91dd-e0b9-4a26-a0c4-9b72fd3b6424 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:09.253304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e91dd-e0b9-4a26-a0c4-9b72fd3b6424 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:09.253608 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e91dd-e0b9-4a26-a0c4-9b72fd3b6424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:09.254975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e91dd-e0b9-4a26-a0c4-9b72fd3b6424 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:09.256313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:09.263539 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 551/1094] 10.222.0.27 () {56 vars in 1115 bytes} [Thu Sep 8 15:56:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:56:09.269407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e91dd-e0b9-4a26-a0c4-9b72fd3b6424 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:09.307536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfee79-4da4-41ad-8328-2dacebf39a50 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:09.341334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2861c9a-db4e-459d-98d0-9b977dd5e8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:09.341962 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2861c9a-db4e-459d-98d0-9b977dd5e8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:09.342239 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2861c9a-db4e-459d-98d0-9b977dd5e8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:09.342505 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2861c9a-db4e-459d-98d0-9b977dd5e8b7 None admin] PATH_INFO: `/v3/projects/037ba1f2c61344058dba200c677d9a37` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:09.344027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2861c9a-db4e-459d-98d0-9b977dd5e8b7 None admin] RBAC: Authorizing `identity:get_project(project_id=037ba1f2c61344058dba200c677d9a37)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:09.345841 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2861c9a-db4e-459d-98d0-9b977dd5e8b7 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:09.347824 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 544/1095] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:56:09 2022] GET /identity/v3/projects/037ba1f2c61344058dba200c677d9a37 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:09.353900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2861c9a-db4e-459d-98d0-9b977dd5e8b7 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:09.367396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e91dd-e0b9-4a26-a0c4-9b72fd3b6424 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:09.378273 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:09.400248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2795b-fd06-468f-9ef5-d40f1044f488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:09.400876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2795b-fd06-468f-9ef5-d40f1044f488 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:09.401144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2795b-fd06-468f-9ef5-d40f1044f488 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:09.401409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2795b-fd06-468f-9ef5-d40f1044f488 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:09.411744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2795b-fd06-468f-9ef5-d40f1044f488 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:09.413185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2795b-fd06-468f-9ef5-d40f1044f488 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-4ad2795b-fd06-468f-9ef5-d40f1044f488 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:56:09.426979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:09.428070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:56:09.428985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:56:09.428985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:56:09.428985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:56:09.428985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 552/1096] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:56:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 15:56:09.433947 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2795b-fd06-468f-9ef5-d40f1044f488 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:09.455564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2861c9a-db4e-459d-98d0-9b977dd5e8b7 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:09.481891 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:09.489977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab3592-1a76-4896-b37b-4cfbe40e5a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:09.490585 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab3592-1a76-4896-b37b-4cfbe40e5a27 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:09.490854 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab3592-1a76-4896-b37b-4cfbe40e5a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:09.491118 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab3592-1a76-4896-b37b-4cfbe40e5a27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:09.491868 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab3592-1a76-4896-b37b-4cfbe40e5a27 None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=037ba1f2c61344058dba200c677d9a37, user.id=4e50afe7c25d40c5b12ad33ae8a4fa26) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:09.492168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab3592-1a76-4896-b37b-4cfbe40e5a27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:09.493586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab3592-1a76-4896-b37b-4cfbe40e5a27 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:09.499256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 545/1097] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:56:09 2022] GET /identity/v3/role_assignments?user.id=4e50afe7c25d40c5b12ad33ae8a4fa26&scope.project.id=037ba1f2c61344058dba200c677d9a37&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:56:09.537427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2795b-fd06-468f-9ef5-d40f1044f488 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:09.575396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb0d22-b752-4414-b04e-619295fa4a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:09.576054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb0d22-b752-4414-b04e-619295fa4a74 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:09.576326 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb0d22-b752-4414-b04e-619295fa4a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:09.576570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fb0d22-b752-4414-b04e-619295fa4a74 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:09.577245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb0d22-b752-4414-b04e-619295fa4a74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:09.577542 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb0d22-b752-4414-b04e-619295fa4a74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:09.579017 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fb0d22-b752-4414-b04e-619295fa4a74 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:09.593177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 553/1098] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:56:09.599209 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab3592-1a76-4896-b37b-4cfbe40e5a27 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:09.694057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:09.706883 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab3592-1a76-4896-b37b-4cfbe40e5a27 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:09.740369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495a7b2-6441-4c2e-aba3-56d37462e02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:09.740997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495a7b2-6441-4c2e-aba3-56d37462e02c None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:09.741264 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495a7b2-6441-4c2e-aba3-56d37462e02c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:09.741531 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495a7b2-6441-4c2e-aba3-56d37462e02c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:09.748464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495a7b2-6441-4c2e-aba3-56d37462e02c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:09.749888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495a7b2-6441-4c2e-aba3-56d37462e02c None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-c495a7b2-6441-4c2e-aba3-56d37462e02c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:56:09.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:09.762314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:09.763288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:09.763288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:09.763288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:56:09.763288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:56:09.763288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:56:09.763288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:56:09.763288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:56:09.763288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:56:09.763288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 15:56:09.763288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:09.763288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 546/1099] 10.222.0.27 () {56 vars in 1093 bytes} [Thu Sep 8 15:56:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:09.768503 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb0d22-b752-4414-b04e-619295fa4a74 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:09.869416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fb0d22-b752-4414-b04e-619295fa4a74 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:09.879008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:09.904612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebedf09-833f-430b-b9a8-e0c434aa4ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:09.905210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebedf09-833f-430b-b9a8-e0c434aa4ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:09.905479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebedf09-833f-430b-b9a8-e0c434aa4ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:09.905741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebedf09-833f-430b-b9a8-e0c434aa4ea7 None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:09.906460 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebedf09-833f-430b-b9a8-e0c434aa4ea7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:09.906754 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebedf09-833f-430b-b9a8-e0c434aa4ea7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:09.908257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebedf09-833f-430b-b9a8-e0c434aa4ea7 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:09.925238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 554/1100] 10.222.0.27 () {56 vars in 1133 bytes} [Thu Sep 8 15:56:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:09.931147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495a7b2-6441-4c2e-aba3-56d37462e02c None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:10.034172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495a7b2-6441-4c2e-aba3-56d37462e02c None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:10.069403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc43ff-c07c-496c-b888-b410c4b4a302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:10.070012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc43ff-c07c-496c-b888-b410c4b4a302 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:10.070282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc43ff-c07c-496c-b888-b410c4b4a302 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:10.070544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc43ff-c07c-496c-b888-b410c4b4a302 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:10.087869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:10.139657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc43ff-c07c-496c-b888-b410c4b4a302 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:10.141113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc43ff-c07c-496c-b888-b410c4b4a302 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:10.246977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 547/1101] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:56:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:56:10.253013 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebedf09-833f-430b-b9a8-e0c434aa4ea7 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:10.353563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebedf09-833f-430b-b9a8-e0c434aa4ea7 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:10.388259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9275a-7057-4e06-8b25-c03b1ed48b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:10.388915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9275a-7057-4e06-8b25-c03b1ed48b87 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:10.389171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9275a-7057-4e06-8b25-c03b1ed48b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:10.389438 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9275a-7057-4e06-8b25-c03b1ed48b87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:10.404343 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9275a-7057-4e06-8b25-c03b1ed48b87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:10.405802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9275a-7057-4e06-8b25-c03b1ed48b87 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-3ef9275a-7057-4e06-8b25-c03b1ed48b87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:56:10.420956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:10.422074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:56:10.423205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:56:10.423205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:56:10.423205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:56:10.426815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 555/1102] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:56:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:10.428097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc43ff-c07c-496c-b888-b410c4b4a302 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:10.476196 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:10.527702 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc43ff-c07c-496c-b888-b410c4b4a302 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:10.561982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cea717-970a-49e1-80a0-6f5c3a9448ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:10.562583 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cea717-970a-49e1-80a0-6f5c3a9448ef None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:10.562861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cea717-970a-49e1-80a0-6f5c3a9448ef None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:10.563155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cea717-970a-49e1-80a0-6f5c3a9448ef None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:10.563850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cea717-970a-49e1-80a0-6f5c3a9448ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:10.564151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cea717-970a-49e1-80a0-6f5c3a9448ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:10.565523 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cea717-970a-49e1-80a0-6f5c3a9448ef None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:10.576813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 548/1103] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:10.582593 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9275a-7057-4e06-8b25-c03b1ed48b87 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:10.603995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:10.687486 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9275a-7057-4e06-8b25-c03b1ed48b87 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:10.723910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d15c1-0e04-413d-8b04-4d70ff277efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:10.724535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d15c1-0e04-413d-8b04-4d70ff277efd None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:10.724802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d15c1-0e04-413d-8b04-4d70ff277efd None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:10.725065 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d15c1-0e04-413d-8b04-4d70ff277efd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:10.731569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d15c1-0e04-413d-8b04-4d70ff277efd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:10.733007 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d15c1-0e04-413d-8b04-4d70ff277efd None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:10.744146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-4f1d15c1-0e04-413d-8b04-4d70ff277efd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:56:10.744146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:10.744146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:10.744146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:10.744146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:10.744146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:10.744146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:10.744146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:10.744146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:10.744146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:10.744146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:10.744146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:10.744146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:56:10.744146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:56:10.744146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:10.745260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:10.746183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:56:10.746183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:56:10.746183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:56:10.746183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:56:10.746183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:56:10.746183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:56:10.746183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:56:10.746183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:56:10.746183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 556/1104] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:56:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:10.751157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cea717-970a-49e1-80a0-6f5c3a9448ef None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:10.853170 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cea717-970a-49e1-80a0-6f5c3a9448ef None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:10.861998 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:10.886874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16604053-ddbf-4ba6-b69d-eb55f9aef445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:10.887477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16604053-ddbf-4ba6-b69d-eb55f9aef445 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:10.887767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16604053-ddbf-4ba6-b69d-eb55f9aef445 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:10.888029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16604053-ddbf-4ba6-b69d-eb55f9aef445 None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:10.888715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16604053-ddbf-4ba6-b69d-eb55f9aef445 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:10.889012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16604053-ddbf-4ba6-b69d-eb55f9aef445 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:10.890431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16604053-ddbf-4ba6-b69d-eb55f9aef445 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:10.902113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 549/1105] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:56:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:10.908406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d15c1-0e04-413d-8b04-4d70ff277efd None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:10.986705 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:11.010405 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d15c1-0e04-413d-8b04-4d70ff277efd None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:11.044716 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9b414c-c562-4871-9ab4-214cd44f73e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:11.045303 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b414c-c562-4871-9ab4-214cd44f73e2 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:11.045570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b414c-c562-4871-9ab4-214cd44f73e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:11.045837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9b414c-c562-4871-9ab4-214cd44f73e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:11.046552 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b414c-c562-4871-9ab4-214cd44f73e2 None admin] RBAC: Adding query filter params (role.id=f752d851596f48a3bb97ad5d8f09d763, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=6e9209e77ce44b128e25a8ddcf78f8b5) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:11.046848 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b414c-c562-4871-9ab4-214cd44f73e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:11.048332 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9b414c-c562-4871-9ab4-214cd44f73e2 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:11.054183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 557/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:56:10 2022] GET /identity/v3/role_assignments?user.id=6e9209e77ce44b128e25a8ddcf78f8b5&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=f752d851596f48a3bb97ad5d8f09d763 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:11.683512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e7059-6c17-4efe-abae-cad7483cbeba None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:11.684086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e7059-6c17-4efe-abae-cad7483cbeba None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:11.684086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e7059-6c17-4efe-abae-cad7483cbeba None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:11.685407 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 550/1107] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:11.691134 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ebd2b-cebf-4572-bbe5-5fcb01d84cde None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:11.691615 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ebd2b-cebf-4572-bbe5-5fcb01d84cde None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:11.691782 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ebd2b-cebf-4572-bbe5-5fcb01d84cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:11.711760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-274ebd2b-cebf-4572-bbe5-5fcb01d84cde None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:11.712953 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-274ebd2b-cebf-4572-bbe5-5fcb01d84cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:11.780469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 558/1108] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:12.046392 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e7059-6c17-4efe-abae-cad7483cbeba None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:12.157080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e7059-6c17-4efe-abae-cad7483cbeba None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:12.194583 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2b65b-f95e-4534-a27c-a891d22b645a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:12.195215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2b65b-f95e-4534-a27c-a891d22b645a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:12.195483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2b65b-f95e-4534-a27c-a891d22b645a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:12.195786 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2b65b-f95e-4534-a27c-a891d22b645a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:12.196928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2b65b-f95e-4534-a27c-a891d22b645a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:12.198411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee2b65b-f95e-4534-a27c-a891d22b645a None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:12.200033 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 551/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:56:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:12.206121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ebd2b-cebf-4572-bbe5-5fcb01d84cde None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:12.281017 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:12.313904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ebd2b-cebf-4572-bbe5-5fcb01d84cde None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:12.351417 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b9330e-9d29-4374-b75b-1e649209d3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:12.352070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9330e-9d29-4374-b75b-1e649209d3b7 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:12.352318 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9330e-9d29-4374-b75b-1e649209d3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:12.352583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b9330e-9d29-4374-b75b-1e649209d3b7 None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:12.353230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b9330e-9d29-4374-b75b-1e649209d3b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:12.354693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b9330e-9d29-4374-b75b-1e649209d3b7 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:12.381395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 559/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:56:12 2022] POST /identity/v3/users => generated 286 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:56:14.521420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d504797-9e12-4627-abce-e99826be80e5 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:14.521931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d504797-9e12-4627-abce-e99826be80e5 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:14.521931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d504797-9e12-4627-abce-e99826be80e5 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:14.524043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 552/1111] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:14.532949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c2d1c-e361-4b16-8463-d7c65a91ca74 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:14.533342 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c2d1c-e361-4b16-8463-d7c65a91ca74 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:14.533590 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c2d1c-e361-4b16-8463-d7c65a91ca74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:14.559011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-5f4c2d1c-e361-4b16-8463-d7c65a91ca74 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:14.560457 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5f4c2d1c-e361-4b16-8463-d7c65a91ca74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:14.630679 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 560/1112] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:14.905464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d504797-9e12-4627-abce-e99826be80e5 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:14.946863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:15.021575 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d504797-9e12-4627-abce-e99826be80e5 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:15.061722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a48b9-90a0-46d9-b95c-063d42a624b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:15.062350 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a48b9-90a0-46d9-b95c-063d42a624b1 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:15.062620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a48b9-90a0-46d9-b95c-063d42a624b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:15.062885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a48b9-90a0-46d9-b95c-063d42a624b1 None admin] PATH_INFO: `/v3/roles/bf98c97818504bc48e3bd1b2aef9d3c3` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:15.064296 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a48b9-90a0-46d9-b95c-063d42a624b1 None admin] RBAC: Authorizing `identity:get_role(role_id=bf98c97818504bc48e3bd1b2aef9d3c3)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:15.065714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a48b9-90a0-46d9-b95c-063d42a624b1 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:15.067056 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 553/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:56:14 2022] GET /identity/v3/roles/bf98c97818504bc48e3bd1b2aef9d3c3 => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:15.073284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c2d1c-e361-4b16-8463-d7c65a91ca74 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:15.134339 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:15.179065 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c2d1c-e361-4b16-8463-d7c65a91ca74 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:15.217641 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7e89f-9794-4df0-9078-b01cf8d247b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:15.218267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7e89f-9794-4df0-9078-b01cf8d247b1 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:15.218537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7e89f-9794-4df0-9078-b01cf8d247b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:15.218804 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7e89f-9794-4df0-9078-b01cf8d247b1 None admin] PATH_INFO: `/v3/users/d11f17794ec549bcb0080c58907b58aa` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:15.232245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7e89f-9794-4df0-9078-b01cf8d247b1 None admin] RBAC: Authorizing `identity:get_user(user_id=d11f17794ec549bcb0080c58907b58aa)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:15.233688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf7e89f-9794-4df0-9078-b01cf8d247b1 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:15.235260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 561/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:56:15 2022] GET /identity/v3/users/d11f17794ec549bcb0080c58907b58aa => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:15.241378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a48b9-90a0-46d9-b95c-063d42a624b1 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:15.358972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a48b9-90a0-46d9-b95c-063d42a624b1 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:15.362035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:15.394627 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ec507-9932-4c73-90ae-4b5514f61e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:15.395219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ec507-9932-4c73-90ae-4b5514f61e29 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:15.395506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ec507-9932-4c73-90ae-4b5514f61e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:15.395787 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ec507-9932-4c73-90ae-4b5514f61e29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:15.462776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ec507-9932-4c73-90ae-4b5514f61e29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:15.464169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ec507-9932-4c73-90ae-4b5514f61e29 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:15.562199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 554/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:56:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:56:15.569053 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7e89f-9794-4df0-9078-b01cf8d247b1 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:15.696339 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf7e89f-9794-4df0-9078-b01cf8d247b1 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:15.713447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:15.732349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b681a0a-3d27-4cff-b6dd-43eba6891b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:15.732965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b681a0a-3d27-4cff-b6dd-43eba6891b34 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:15.733218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b681a0a-3d27-4cff-b6dd-43eba6891b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:15.733501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b681a0a-3d27-4cff-b6dd-43eba6891b34 None admin] PATH_INFO: `/v3/projects/037ba1f2c61344058dba200c677d9a37` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:15.734644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b681a0a-3d27-4cff-b6dd-43eba6891b34 None admin] RBAC: Authorizing `identity:get_project(project_id=037ba1f2c61344058dba200c677d9a37)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:15.736115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b681a0a-3d27-4cff-b6dd-43eba6891b34 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:15.737725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 562/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:56:15 2022] GET /identity/v3/projects/037ba1f2c61344058dba200c677d9a37 => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:15.744303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ec507-9932-4c73-90ae-4b5514f61e29 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:15.747506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:15.849732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ec507-9932-4c73-90ae-4b5514f61e29 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:15.886339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052790c-a963-4d44-b752-d2382d7255f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:15.886920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052790c-a963-4d44-b752-d2382d7255f3 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:15.887187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052790c-a963-4d44-b752-d2382d7255f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:15.887488 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052790c-a963-4d44-b752-d2382d7255f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:15.888192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052790c-a963-4d44-b752-d2382d7255f3 None admin] RBAC: Adding query filter params (role.id=bf98c97818504bc48e3bd1b2aef9d3c3, scope.project.id=037ba1f2c61344058dba200c677d9a37, user.id=d11f17794ec549bcb0080c58907b58aa) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:15.888486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052790c-a963-4d44-b752-d2382d7255f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:15.889907 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052790c-a963-4d44-b752-d2382d7255f3 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:15.895365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 555/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:56:15 2022] GET /identity/v3/role_assignments?user.id=d11f17794ec549bcb0080c58907b58aa&scope.project.id=037ba1f2c61344058dba200c677d9a37&role.id=bf98c97818504bc48e3bd1b2aef9d3c3 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:17.968966 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da38b1-055a-45f5-95d8-56d521a3a4d6 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:17.969593 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da38b1-055a-45f5-95d8-56d521a3a4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:17.969593 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da38b1-055a-45f5-95d8-56d521a3a4d6 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:17.970899 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 563/1118] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:17.977408 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e1e05-0525-458c-9027-d0ca70865e4a None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:17.978128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e1e05-0525-458c-9027-d0ca70865e4a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:17.978287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e1e05-0525-458c-9027-d0ca70865e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:18.006705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-684e1e05-0525-458c-9027-d0ca70865e4a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:18.007988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-684e1e05-0525-458c-9027-d0ca70865e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:18.080934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:18.089556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 556/1119] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:18.361070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da38b1-055a-45f5-95d8-56d521a3a4d6 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:18.474434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da38b1-055a-45f5-95d8-56d521a3a4d6 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:18.518334 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc2f73-eeda-40a4-974b-b6dad4f603d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:18.518961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc2f73-eeda-40a4-974b-b6dad4f603d2 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:18.519238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc2f73-eeda-40a4-974b-b6dad4f603d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:18.519534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc2f73-eeda-40a4-974b-b6dad4f603d2 None admin] PATH_INFO: `/v3/roles/bf98c97818504bc48e3bd1b2aef9d3c3` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:18.521080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc2f73-eeda-40a4-974b-b6dad4f603d2 None admin] RBAC: Authorizing `identity:get_role(role_id=bf98c97818504bc48e3bd1b2aef9d3c3)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:18.522475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc2f73-eeda-40a4-974b-b6dad4f603d2 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:18.523903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 564/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:56:18 2022] GET /identity/v3/roles/bf98c97818504bc48e3bd1b2aef9d3c3 => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:18.532732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e1e05-0525-458c-9027-d0ca70865e4a None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:18.635950 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e1e05-0525-458c-9027-d0ca70865e4a None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:18.677447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01f56a-f375-4f0a-9f33-649209453c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:18.678085 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01f56a-f375-4f0a-9f33-649209453c90 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:18.678363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01f56a-f375-4f0a-9f33-649209453c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:18.678633 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a01f56a-f375-4f0a-9f33-649209453c90 None admin] PATH_INFO: `/v3/users/d11f17794ec549bcb0080c58907b58aa` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:18.679839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01f56a-f375-4f0a-9f33-649209453c90 None admin] RBAC: Authorizing `identity:get_user(user_id=d11f17794ec549bcb0080c58907b58aa)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:18.681273 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a01f56a-f375-4f0a-9f33-649209453c90 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:18.682863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 557/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:56:18 2022] GET /identity/v3/users/d11f17794ec549bcb0080c58907b58aa => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:18.688703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc2f73-eeda-40a4-974b-b6dad4f603d2 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:18.743347 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:18.795675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc2f73-eeda-40a4-974b-b6dad4f603d2 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:18.831694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2dbd18-758a-4ee1-a708-7740aaa003bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:18.832311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dbd18-758a-4ee1-a708-7740aaa003bd None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:18.832586 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dbd18-758a-4ee1-a708-7740aaa003bd None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:18.832853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dbd18-758a-4ee1-a708-7740aaa003bd None admin] PATH_INFO: `/v3/projects/037ba1f2c61344058dba200c677d9a37` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:18.834031 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dbd18-758a-4ee1-a708-7740aaa003bd None admin] RBAC: Authorizing `identity:get_project(project_id=037ba1f2c61344058dba200c677d9a37)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:18.835861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2dbd18-758a-4ee1-a708-7740aaa003bd None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:18.837487 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 565/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:56:18 2022] GET /identity/v3/projects/037ba1f2c61344058dba200c677d9a37 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:18.844260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01f56a-f375-4f0a-9f33-649209453c90 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:18.921329 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:18.959017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a01f56a-f375-4f0a-9f33-649209453c90 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:18.995231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9000d-d8c2-4af9-bf62-3312f0785a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:18.995878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9000d-d8c2-4af9-bf62-3312f0785a0f None admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:18.996163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9000d-d8c2-4af9-bf62-3312f0785a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:18.996429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9000d-d8c2-4af9-bf62-3312f0785a0f None admin] PATH_INFO: `/v3/projects/037ba1f2c61344058dba200c677d9a37/users/d11f17794ec549bcb0080c58907b58aa/roles/bf98c97818504bc48e3bd1b2aef9d3c3` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:18.998634 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9000d-d8c2-4af9-bf62-3312f0785a0f None admin] RBAC: Authorizing `identity:create_grant(project_id=037ba1f2c61344058dba200c677d9a37, user_id=d11f17794ec549bcb0080c58907b58aa, role_id=bf98c97818504bc48e3bd1b2aef9d3c3)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:19.000093 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9000d-d8c2-4af9-bf62-3312f0785a0f None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:19.015343 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 558/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:56:18 2022] PUT /identity/v3/projects/037ba1f2c61344058dba200c677d9a37/users/d11f17794ec549bcb0080c58907b58aa/roles/bf98c97818504bc48e3bd1b2aef9d3c3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:56:21.151268 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7272582-daa0-4598-a992-9f9ef9f4e400 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:21.152181 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7272582-daa0-4598-a992-9f9ef9f4e400 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:21.152181 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7272582-daa0-4598-a992-9f9ef9f4e400 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:21.154367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 566/1124] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:21.160713 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508bbd6-af99-43ba-8dbe-e620f210e809 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:21.161403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508bbd6-af99-43ba-8dbe-e620f210e809 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:21.161537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508bbd6-af99-43ba-8dbe-e620f210e809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:21.185137 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-1508bbd6-af99-43ba-8dbe-e620f210e809 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:21.186320 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1508bbd6-af99-43ba-8dbe-e620f210e809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:21.271516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 559/1125] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:21.548230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7272582-daa0-4598-a992-9f9ef9f4e400 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:21.665145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7272582-daa0-4598-a992-9f9ef9f4e400 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:21.686242 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:21.702919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a880e-fea1-4af3-b987-170c5212b5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:21.703584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a880e-fea1-4af3-b987-170c5212b5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:21.703858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a880e-fea1-4af3-b987-170c5212b5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:21.704121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a880e-fea1-4af3-b987-170c5212b5f8 None admin] PATH_INFO: `/v3/roles/bf98c97818504bc48e3bd1b2aef9d3c3` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:21.705621 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a880e-fea1-4af3-b987-170c5212b5f8 None admin] RBAC: Authorizing `identity:get_role(role_id=bf98c97818504bc48e3bd1b2aef9d3c3)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:21.707040 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a880e-fea1-4af3-b987-170c5212b5f8 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:21.708395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 567/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:56:21 2022] GET /identity/v3/roles/bf98c97818504bc48e3bd1b2aef9d3c3 => generated 223 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:21.714686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508bbd6-af99-43ba-8dbe-e620f210e809 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:21.771889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:21.819542 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508bbd6-af99-43ba-8dbe-e620f210e809 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:21.856755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce1ee4-bd6e-4d84-9aa6-8c37890c43e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:21.857383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce1ee4-bd6e-4d84-9aa6-8c37890c43e9 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:21.857656 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce1ee4-bd6e-4d84-9aa6-8c37890c43e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:21.857931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce1ee4-bd6e-4d84-9aa6-8c37890c43e9 None admin] PATH_INFO: `/v3/users/d11f17794ec549bcb0080c58907b58aa` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:21.859175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce1ee4-bd6e-4d84-9aa6-8c37890c43e9 None admin] RBAC: Authorizing `identity:get_user(user_id=d11f17794ec549bcb0080c58907b58aa)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:21.860648 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce1ee4-bd6e-4d84-9aa6-8c37890c43e9 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:21.862300 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 560/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:56:21 2022] GET /identity/v3/users/d11f17794ec549bcb0080c58907b58aa => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:21.871707 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a880e-fea1-4af3-b987-170c5212b5f8 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:21.990129 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a880e-fea1-4af3-b987-170c5212b5f8 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:22.025769 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3d95b-1b99-4496-a815-b64cd1dabccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:22.026359 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3d95b-1b99-4496-a815-b64cd1dabccf None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:22.026634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3d95b-1b99-4496-a815-b64cd1dabccf None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:22.026901 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3d95b-1b99-4496-a815-b64cd1dabccf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:22.085713 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:22.100126 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3d95b-1b99-4496-a815-b64cd1dabccf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:22.101590 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3d95b-1b99-4496-a815-b64cd1dabccf None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:22.202945 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 568/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:56:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:56:22.209088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce1ee4-bd6e-4d84-9aa6-8c37890c43e9 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:22.311682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce1ee4-bd6e-4d84-9aa6-8c37890c43e9 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:22.337088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:22.351565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacd662b-c844-42e5-9cd7-732b87f48d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:22.352194 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd662b-c844-42e5-9cd7-732b87f48d05 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:22.352467 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd662b-c844-42e5-9cd7-732b87f48d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:22.352730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacd662b-c844-42e5-9cd7-732b87f48d05 None admin] PATH_INFO: `/v3/projects/037ba1f2c61344058dba200c677d9a37` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:22.353843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd662b-c844-42e5-9cd7-732b87f48d05 None admin] RBAC: Authorizing `identity:get_project(project_id=037ba1f2c61344058dba200c677d9a37)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:22.355274 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacd662b-c844-42e5-9cd7-732b87f48d05 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:22.356952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 561/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:56:22 2022] GET /identity/v3/projects/037ba1f2c61344058dba200c677d9a37 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:22.363211 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3d95b-1b99-4496-a815-b64cd1dabccf None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:22.455262 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:22.469105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3d95b-1b99-4496-a815-b64cd1dabccf None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:22.504585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366f9dca-6c40-4bc4-9390-dd5fff544fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:22.505178 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f9dca-6c40-4bc4-9390-dd5fff544fed None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:22.505451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f9dca-6c40-4bc4-9390-dd5fff544fed None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:22.505713 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366f9dca-6c40-4bc4-9390-dd5fff544fed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:22.506426 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f9dca-6c40-4bc4-9390-dd5fff544fed None admin] RBAC: Adding query filter params (role.id=bf98c97818504bc48e3bd1b2aef9d3c3, scope.project.id=037ba1f2c61344058dba200c677d9a37, user.id=d11f17794ec549bcb0080c58907b58aa) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:22.506726 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f9dca-6c40-4bc4-9390-dd5fff544fed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:22.508182 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366f9dca-6c40-4bc4-9390-dd5fff544fed None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:22.514040 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 569/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:56:22 2022] GET /identity/v3/role_assignments?user.id=d11f17794ec549bcb0080c58907b58aa&scope.project.id=037ba1f2c61344058dba200c677d9a37&role.id=bf98c97818504bc48e3bd1b2aef9d3c3 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:24.616630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664610d-2520-446f-8f97-e2bcc75f9440 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:24.617496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664610d-2520-446f-8f97-e2bcc75f9440 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:24.617496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664610d-2520-446f-8f97-e2bcc75f9440 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:24.619763 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 562/1131] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:24.626003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f6996-4815-441c-a103-45d3efdd88af None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:24.626838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f6996-4815-441c-a103-45d3efdd88af None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:24.626948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f6996-4815-441c-a103-45d3efdd88af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:24.647626 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-bb4f6996-4815-441c-a103-45d3efdd88af None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:24.648800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-bb4f6996-4815-441c-a103-45d3efdd88af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:24.721362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 570/1132] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:24.997318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664610d-2520-446f-8f97-e2bcc75f9440 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:25.109146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664610d-2520-446f-8f97-e2bcc75f9440 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:25.143460 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74215161-fbcf-4bb4-8da6-7eee342a38f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:25.144087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74215161-fbcf-4bb4-8da6-7eee342a38f2 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:25.144357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74215161-fbcf-4bb4-8da6-7eee342a38f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:25.144627 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74215161-fbcf-4bb4-8da6-7eee342a38f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:25.145834 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74215161-fbcf-4bb4-8da6-7eee342a38f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:25.147260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74215161-fbcf-4bb4-8da6-7eee342a38f2 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:25.148841 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 563/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:56:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:25.154765 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f6996-4815-441c-a103-45d3efdd88af None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:25.245086 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:25.265830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f6996-4815-441c-a103-45d3efdd88af None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:25.306853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e80ed-55f7-4db9-bc18-442cfdaeb858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:25.307477 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e80ed-55f7-4db9-bc18-442cfdaeb858 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:25.307757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e80ed-55f7-4db9-bc18-442cfdaeb858 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:25.308020 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e80ed-55f7-4db9-bc18-442cfdaeb858 None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:25.308673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e80ed-55f7-4db9-bc18-442cfdaeb858 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:25.310118 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e80ed-55f7-4db9-bc18-442cfdaeb858 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:25.342367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 571/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Sep 8 15:56:25 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:56:27.486133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ff1b-8a57-47dc-931f-bf10778b912a None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:27.486950 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ff1b-8a57-47dc-931f-bf10778b912a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:27.486950 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ff1b-8a57-47dc-931f-bf10778b912a None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:27.489139 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 564/1135] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:27.495504 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85428ca-6b81-4265-9656-b30787f15f36 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:27.495934 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85428ca-6b81-4265-9656-b30787f15f36 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:27.496071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85428ca-6b81-4265-9656-b30787f15f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:27.517468 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-e85428ca-6b81-4265-9656-b30787f15f36 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:27.518656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e85428ca-6b81-4265-9656-b30787f15f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:27.590942 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 572/1136] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:27.854165 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227ff1b-8a57-47dc-931f-bf10778b912a None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:27.918861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:27.966301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227ff1b-8a57-47dc-931f-bf10778b912a None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:28.001599 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45371af2-bfea-4aff-83e9-0c837261afef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:28.002212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45371af2-bfea-4aff-83e9-0c837261afef None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:28.002484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45371af2-bfea-4aff-83e9-0c837261afef None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:28.002748 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45371af2-bfea-4aff-83e9-0c837261afef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:28.004032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45371af2-bfea-4aff-83e9-0c837261afef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:28.005468 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45371af2-bfea-4aff-83e9-0c837261afef None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:28.007032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 565/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:56:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:28.016120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85428ca-6b81-4265-9656-b30787f15f36 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:28.082380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:28.124467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85428ca-6b81-4265-9656-b30787f15f36 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:28.159293 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8f487-2e12-41d1-94ab-3604b2a452ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:28.159965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8f487-2e12-41d1-94ab-3604b2a452ee None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:28.160235 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8f487-2e12-41d1-94ab-3604b2a452ee None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:28.160500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8f487-2e12-41d1-94ab-3604b2a452ee None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:28.161149 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8f487-2e12-41d1-94ab-3604b2a452ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:28.162574 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8f487-2e12-41d1-94ab-3604b2a452ee None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:28.186688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 573/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:56:28 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:56:30.337620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cbf3a-9394-42fa-9857-c59ce279a514 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:30.338175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cbf3a-9394-42fa-9857-c59ce279a514 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:30.338175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cbf3a-9394-42fa-9857-c59ce279a514 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:30.339508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 566/1139] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:30.345134 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31b73b-893c-400e-9816-3bf7ac4b895c None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:30.345550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31b73b-893c-400e-9816-3bf7ac4b895c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:30.345660 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31b73b-893c-400e-9816-3bf7ac4b895c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:30.370136 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-cf31b73b-893c-400e-9816-3bf7ac4b895c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:30.371319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-cf31b73b-893c-400e-9816-3bf7ac4b895c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:30.450911 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 574/1140] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:30.723028 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cbf3a-9394-42fa-9857-c59ce279a514 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:30.835610 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cbf3a-9394-42fa-9857-c59ce279a514 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:30.862978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:30.874390 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3457af-303f-4aa8-b88f-832c8b6eb30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:30.875375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3457af-303f-4aa8-b88f-832c8b6eb30a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:30.875781 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3457af-303f-4aa8-b88f-832c8b6eb30a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:30.876156 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3457af-303f-4aa8-b88f-832c8b6eb30a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:30.894712 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3457af-303f-4aa8-b88f-832c8b6eb30a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-9d3457af-303f-4aa8-b88f-832c8b6eb30a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:56:30.899619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:30.900758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:30.901905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:30.901905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:30.901905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:30.901905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:30.901905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:56:30.901905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:56:30.901905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:56:30.901905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:56:30.901905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:56:30.901905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:56:30.901905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:30.901905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:30.901905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 567/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:56:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:30.906822 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31b73b-893c-400e-9816-3bf7ac4b895c None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:30.962228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:31.024486 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31b73b-893c-400e-9816-3bf7ac4b895c None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:31.060283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97783bc-c4dd-4b7a-aac8-6a41f80ad9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:31.060893 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97783bc-c4dd-4b7a-aac8-6a41f80ad9de None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:31.061167 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97783bc-c4dd-4b7a-aac8-6a41f80ad9de None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:31.061433 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97783bc-c4dd-4b7a-aac8-6a41f80ad9de None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:31.062103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97783bc-c4dd-4b7a-aac8-6a41f80ad9de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:31.062402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97783bc-c4dd-4b7a-aac8-6a41f80ad9de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:31.063824 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97783bc-c4dd-4b7a-aac8-6a41f80ad9de None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:31.072132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 575/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:56:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:31.081071 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3457af-303f-4aa8-b88f-832c8b6eb30a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:31.188069 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3457af-303f-4aa8-b88f-832c8b6eb30a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:31.223680 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69b8fb-c1b8-4e85-b31e-ba4e45297e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:31.224312 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69b8fb-c1b8-4e85-b31e-ba4e45297e7a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:31.224593 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69b8fb-c1b8-4e85-b31e-ba4e45297e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:31.224861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb69b8fb-c1b8-4e85-b31e-ba4e45297e7a None admin] PATH_INFO: `/v3/users/ca75f5c8924648f696e19ecfbe222503` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:31.239875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69b8fb-c1b8-4e85-b31e-ba4e45297e7a None admin] RBAC: Authorizing `identity:get_user(user_id=ca75f5c8924648f696e19ecfbe222503)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:31.241326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb69b8fb-c1b8-4e85-b31e-ba4e45297e7a None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:31.242965 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 568/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:56:31 2022] GET /identity/v3/users/ca75f5c8924648f696e19ecfbe222503 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:31.249644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97783bc-c4dd-4b7a-aac8-6a41f80ad9de None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:31.372507 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97783bc-c4dd-4b7a-aac8-6a41f80ad9de None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:31.387548 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:31.417018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095c0f9-cf0a-4a1d-a929-5cfac21eb25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:31.417620 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095c0f9-cf0a-4a1d-a929-5cfac21eb25a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:31.417892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095c0f9-cf0a-4a1d-a929-5cfac21eb25a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:31.418253 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095c0f9-cf0a-4a1d-a929-5cfac21eb25a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:31.486933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095c0f9-cf0a-4a1d-a929-5cfac21eb25a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:31.488594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095c0f9-cf0a-4a1d-a929-5cfac21eb25a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:31.588462 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 576/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:56:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:56:31.598099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69b8fb-c1b8-4e85-b31e-ba4e45297e7a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:31.654425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:31.717946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb69b8fb-c1b8-4e85-b31e-ba4e45297e7a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:31.753666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc95f738-e82d-4ee0-8d9c-18ed739cc147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:31.754282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95f738-e82d-4ee0-8d9c-18ed739cc147 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:31.754552 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95f738-e82d-4ee0-8d9c-18ed739cc147 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:31.754823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc95f738-e82d-4ee0-8d9c-18ed739cc147 None admin] PATH_INFO: `/v3/projects/cfa21f69b56945a189d7f7197a105e20` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:31.756017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95f738-e82d-4ee0-8d9c-18ed739cc147 None admin] RBAC: Authorizing `identity:get_project(project_id=cfa21f69b56945a189d7f7197a105e20)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:31.757440 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc95f738-e82d-4ee0-8d9c-18ed739cc147 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:31.759101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 569/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:56:31 2022] GET /identity/v3/projects/cfa21f69b56945a189d7f7197a105e20 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:31.765589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095c0f9-cf0a-4a1d-a929-5cfac21eb25a None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:31.787929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:31.876338 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095c0f9-cf0a-4a1d-a929-5cfac21eb25a None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:31.914054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f9ba2-23d3-4b8c-8d6e-83dbfff67a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:31.914581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9ba2-23d3-4b8c-8d6e-83dbfff67a3c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:31.914850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9ba2-23d3-4b8c-8d6e-83dbfff67a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:31.915120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9ba2-23d3-4b8c-8d6e-83dbfff67a3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:31.915868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f9ba2-23d3-4b8c-8d6e-83dbfff67a3c None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=cfa21f69b56945a189d7f7197a105e20, user.id=ca75f5c8924648f696e19ecfbe222503) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:31.916172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f9ba2-23d3-4b8c-8d6e-83dbfff67a3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:31.917617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f9ba2-23d3-4b8c-8d6e-83dbfff67a3c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:31.923169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 577/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:56:31 2022] GET /identity/v3/role_assignments?user.id=ca75f5c8924648f696e19ecfbe222503&scope.project.id=cfa21f69b56945a189d7f7197a105e20&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:56:34.001458 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d9787-328c-4acb-affb-02db3c70d062 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:34.002017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d9787-328c-4acb-affb-02db3c70d062 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:34.002017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d9787-328c-4acb-affb-02db3c70d062 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:34.004438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 570/1147] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:34.015596 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff347bf6-a77c-448e-a84e-269759a643dc None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:34.016338 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff347bf6-a77c-448e-a84e-269759a643dc None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:34.016486 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff347bf6-a77c-448e-a84e-269759a643dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:34.039084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-ff347bf6-a77c-448e-a84e-269759a643dc None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:34.040333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ff347bf6-a77c-448e-a84e-269759a643dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:34.125286 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 578/1148] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:34.399234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d9787-328c-4acb-affb-02db3c70d062 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:34.510536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d9787-328c-4acb-affb-02db3c70d062 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:34.525383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:34.547839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a42cc-8b5b-434c-b884-08a8a1cae4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:34.548463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a42cc-8b5b-434c-b884-08a8a1cae4a7 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:34.548734 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a42cc-8b5b-434c-b884-08a8a1cae4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:34.549078 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386a42cc-8b5b-434c-b884-08a8a1cae4a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:34.559399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386a42cc-8b5b-434c-b884-08a8a1cae4a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-386a42cc-8b5b-434c-b884-08a8a1cae4a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:56:34.564252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:34.565403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:34.566671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:34.566671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:34.566671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:34.566671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:34.566671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:56:34.566671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:56:34.566671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:56:34.566671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:56:34.566671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:56:34.566671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:56:34.566671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:34.566671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:34.566671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 571/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:56:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:34.571579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff347bf6-a77c-448e-a84e-269759a643dc None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:34.579748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:34.673542 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff347bf6-a77c-448e-a84e-269759a643dc None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:34.712958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd85f7-ce6b-4c68-a1f7-9b78b1e56d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:34.713566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd85f7-ce6b-4c68-a1f7-9b78b1e56d42 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:34.713839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd85f7-ce6b-4c68-a1f7-9b78b1e56d42 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:34.714105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dd85f7-ce6b-4c68-a1f7-9b78b1e56d42 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:34.714781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd85f7-ce6b-4c68-a1f7-9b78b1e56d42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:34.715083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd85f7-ce6b-4c68-a1f7-9b78b1e56d42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:34.716503 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dd85f7-ce6b-4c68-a1f7-9b78b1e56d42 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:34.725008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 579/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:56:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:34.730952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a42cc-8b5b-434c-b884-08a8a1cae4a7 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:34.832184 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386a42cc-8b5b-434c-b884-08a8a1cae4a7 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:34.867302 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950759a-b66d-4d54-b5ce-3b01bb9ea168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:34.867944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950759a-b66d-4d54-b5ce-3b01bb9ea168 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:34.868215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950759a-b66d-4d54-b5ce-3b01bb9ea168 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:34.868489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950759a-b66d-4d54-b5ce-3b01bb9ea168 None admin] PATH_INFO: `/v3/users/ca75f5c8924648f696e19ecfbe222503` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:34.869638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950759a-b66d-4d54-b5ce-3b01bb9ea168 None admin] RBAC: Authorizing `identity:get_user(user_id=ca75f5c8924648f696e19ecfbe222503)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:34.871062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4950759a-b66d-4d54-b5ce-3b01bb9ea168 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:34.872658 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 572/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:56:34 2022] GET /identity/v3/users/ca75f5c8924648f696e19ecfbe222503 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:34.878744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd85f7-ce6b-4c68-a1f7-9b78b1e56d42 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:34.955320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:34.997430 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dd85f7-ce6b-4c68-a1f7-9b78b1e56d42 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:35.033227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2fd9b-fcd4-4fe7-a35c-bbcdb466c5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:35.033840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2fd9b-fcd4-4fe7-a35c-bbcdb466c5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:35.034110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2fd9b-fcd4-4fe7-a35c-bbcdb466c5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:35.034474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2fd9b-fcd4-4fe7-a35c-bbcdb466c5b9 None admin] PATH_INFO: `/v3/projects/cfa21f69b56945a189d7f7197a105e20` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:35.035763 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2fd9b-fcd4-4fe7-a35c-bbcdb466c5b9 None admin] RBAC: Authorizing `identity:get_project(project_id=cfa21f69b56945a189d7f7197a105e20)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:35.037190 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2fd9b-fcd4-4fe7-a35c-bbcdb466c5b9 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:35.038810 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 580/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:56:34 2022] GET /identity/v3/projects/cfa21f69b56945a189d7f7197a105e20 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:35.044908 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950759a-b66d-4d54-b5ce-3b01bb9ea168 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:35.099480 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:35.160258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950759a-b66d-4d54-b5ce-3b01bb9ea168 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:35.198103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaaf0cb-e985-4201-b69f-4519d75a093d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:35.198793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaaf0cb-e985-4201-b69f-4519d75a093d None admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:35.199066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaaf0cb-e985-4201-b69f-4519d75a093d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:35.199341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaaf0cb-e985-4201-b69f-4519d75a093d None admin] PATH_INFO: `/v3/projects/cfa21f69b56945a189d7f7197a105e20/users/ca75f5c8924648f696e19ecfbe222503/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:35.201287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaaf0cb-e985-4201-b69f-4519d75a093d None admin] RBAC: Authorizing `identity:create_grant(project_id=cfa21f69b56945a189d7f7197a105e20, user_id=ca75f5c8924648f696e19ecfbe222503, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:35.202736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaaf0cb-e985-4201-b69f-4519d75a093d None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:35.222486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 573/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:56:35 2022] PUT /identity/v3/projects/cfa21f69b56945a189d7f7197a105e20/users/ca75f5c8924648f696e19ecfbe222503/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:56:37.307701 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f79af4-dc7a-4020-baa0-7d87199cc991 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:37.308329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f79af4-dc7a-4020-baa0-7d87199cc991 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:37.308329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f79af4-dc7a-4020-baa0-7d87199cc991 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:37.309587 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 581/1154] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:37.316096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a57b1f-32a1-45de-805d-0707bfc75971 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:37.316865 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a57b1f-32a1-45de-805d-0707bfc75971 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:37.316997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a57b1f-32a1-45de-805d-0707bfc75971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:37.348198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-d3a57b1f-32a1-45de-805d-0707bfc75971 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:37.349400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d3a57b1f-32a1-45de-805d-0707bfc75971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:37.427991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 574/1155] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:37.693899 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f79af4-dc7a-4020-baa0-7d87199cc991 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:37.812701 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f79af4-dc7a-4020-baa0-7d87199cc991 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:37.849128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471210d0-7a9b-487b-b766-ec4a2063760d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:37.849735 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471210d0-7a9b-487b-b766-ec4a2063760d None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:37.850002 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471210d0-7a9b-487b-b766-ec4a2063760d None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:37.850265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471210d0-7a9b-487b-b766-ec4a2063760d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:37.853198 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:37.862667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471210d0-7a9b-487b-b766-ec4a2063760d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-471210d0-7a9b-487b-b766-ec4a2063760d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:56:37.867461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:37.868584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:37.869651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:37.869651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:37.869651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:37.869651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:37.869651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:56:37.869651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:56:37.869651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:56:37.869651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:56:37.869651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:56:37.869651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:56:37.869651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:37.869651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:56:37.869651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 582/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:56:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:37.875304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a57b1f-32a1-45de-805d-0707bfc75971 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:37.909173 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:37.978546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a57b1f-32a1-45de-805d-0707bfc75971 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:38.013566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf6c48-0de6-4a3e-92a0-7ff9d420a22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:38.014159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf6c48-0de6-4a3e-92a0-7ff9d420a22a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:38.014431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf6c48-0de6-4a3e-92a0-7ff9d420a22a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:38.014700 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbf6c48-0de6-4a3e-92a0-7ff9d420a22a None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:38.015358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf6c48-0de6-4a3e-92a0-7ff9d420a22a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:38.015687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf6c48-0de6-4a3e-92a0-7ff9d420a22a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:38.017053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbf6c48-0de6-4a3e-92a0-7ff9d420a22a None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:38.025139 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 575/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:56:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:38.032004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471210d0-7a9b-487b-b766-ec4a2063760d None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:38.144222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471210d0-7a9b-487b-b766-ec4a2063760d None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:38.188462 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0648dc1-8061-4b61-88fd-327f38c6b5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:38.189096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0648dc1-8061-4b61-88fd-327f38c6b5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:38.189370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0648dc1-8061-4b61-88fd-327f38c6b5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:38.189643 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0648dc1-8061-4b61-88fd-327f38c6b5b1 None admin] PATH_INFO: `/v3/users/ca75f5c8924648f696e19ecfbe222503` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:38.190879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0648dc1-8061-4b61-88fd-327f38c6b5b1 None admin] RBAC: Authorizing `identity:get_user(user_id=ca75f5c8924648f696e19ecfbe222503)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:38.192367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0648dc1-8061-4b61-88fd-327f38c6b5b1 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:38.193961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 583/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:56:38 2022] GET /identity/v3/users/ca75f5c8924648f696e19ecfbe222503 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:38.200043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf6c48-0de6-4a3e-92a0-7ff9d420a22a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:38.302975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:38.310925 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbf6c48-0de6-4a3e-92a0-7ff9d420a22a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:38.349831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716e6b4-cf48-4784-a71c-9db475d44a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:38.350431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716e6b4-cf48-4784-a71c-9db475d44a8c None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:38.350701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716e6b4-cf48-4784-a71c-9db475d44a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:38.350967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8716e6b4-cf48-4784-a71c-9db475d44a8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:38.432010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716e6b4-cf48-4784-a71c-9db475d44a8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:38.433466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8716e6b4-cf48-4784-a71c-9db475d44a8c None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:38.528579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:38.536422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 576/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:56:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:56:38.545529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0648dc1-8061-4b61-88fd-327f38c6b5b1 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:38.620306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:38.662952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0648dc1-8061-4b61-88fd-327f38c6b5b1 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:38.706088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fc094b-24a0-46d3-b0e4-910bca83148f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:38.706710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc094b-24a0-46d3-b0e4-910bca83148f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:38.706987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc094b-24a0-46d3-b0e4-910bca83148f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:38.707259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fc094b-24a0-46d3-b0e4-910bca83148f None admin] PATH_INFO: `/v3/projects/cfa21f69b56945a189d7f7197a105e20` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:38.708509 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc094b-24a0-46d3-b0e4-910bca83148f None admin] RBAC: Authorizing `identity:get_project(project_id=cfa21f69b56945a189d7f7197a105e20)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:38.710335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fc094b-24a0-46d3-b0e4-910bca83148f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:38.711989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 584/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:56:38 2022] GET /identity/v3/projects/cfa21f69b56945a189d7f7197a105e20 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:38.718345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716e6b4-cf48-4784-a71c-9db475d44a8c None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:38.823542 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8716e6b4-cf48-4784-a71c-9db475d44a8c None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:38.858496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05bcddf-2e28-4066-a509-9555a18d75eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:38.859085 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bcddf-2e28-4066-a509-9555a18d75eb None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:38.859359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bcddf-2e28-4066-a509-9555a18d75eb None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:38.859646 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bcddf-2e28-4066-a509-9555a18d75eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:38.860372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05bcddf-2e28-4066-a509-9555a18d75eb None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=cfa21f69b56945a189d7f7197a105e20, user.id=ca75f5c8924648f696e19ecfbe222503) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:38.860669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05bcddf-2e28-4066-a509-9555a18d75eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:38.862095 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05bcddf-2e28-4066-a509-9555a18d75eb None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:38.867958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 577/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:56:38 2022] GET /identity/v3/role_assignments?user.id=ca75f5c8924648f696e19ecfbe222503&scope.project.id=cfa21f69b56945a189d7f7197a105e20&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:40.981545 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef38a8-8c0c-45dc-986b-6ebbcd7caa66 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:40.982449 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef38a8-8c0c-45dc-986b-6ebbcd7caa66 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:40.982449 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef38a8-8c0c-45dc-986b-6ebbcd7caa66 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:40.984592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 585/1162] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:40.993472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c61ac-ce66-45c5-8b2b-d5952950d78c None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:40.993904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c61ac-ce66-45c5-8b2b-d5952950d78c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:40.994037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c61ac-ce66-45c5-8b2b-d5952950d78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:41.016144 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-253c61ac-ce66-45c5-8b2b-d5952950d78c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:41.017350 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-253c61ac-ce66-45c5-8b2b-d5952950d78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:41.081012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:41.096537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 578/1163] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:41.371303 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef38a8-8c0c-45dc-986b-6ebbcd7caa66 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:41.495752 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef38a8-8c0c-45dc-986b-6ebbcd7caa66 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:41.534271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:41.540365 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9649d-647f-41c2-adcd-1afe7b164bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:41.540964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9649d-647f-41c2-adcd-1afe7b164bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:41.541238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9649d-647f-41c2-adcd-1afe7b164bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:41.541503 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9649d-647f-41c2-adcd-1afe7b164bd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:41.612151 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9649d-647f-41c2-adcd-1afe7b164bd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:41.613558 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be9649d-647f-41c2-adcd-1afe7b164bd2 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:41.729116 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 586/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:56:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:56:41.735461 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c61ac-ce66-45c5-8b2b-d5952950d78c None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:41.835025 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253c61ac-ce66-45c5-8b2b-d5952950d78c None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:41.869739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9641f798-fd33-4451-ad1f-138ae0d39263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:41.870350 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641f798-fd33-4451-ad1f-138ae0d39263 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:41.870622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641f798-fd33-4451-ad1f-138ae0d39263 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:41.870887 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9641f798-fd33-4451-ad1f-138ae0d39263 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:41.877947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641f798-fd33-4451-ad1f-138ae0d39263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:41.879375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9641f798-fd33-4451-ad1f-138ae0d39263 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-9641f798-fd33-4451-ad1f-138ae0d39263 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:41.891169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:41.895778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:41.896904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:41.896904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:41.896904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:41.896904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:41.896904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:41.896904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:41.896904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:41.896904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:41.896904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:41.896904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:41.896904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 08 15:56:41.896904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:56:41.896904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 08 15:56:41.896904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:56:41.896904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 579/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Sep 8 15:56:41 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:41.898982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9649d-647f-41c2-adcd-1afe7b164bd2 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:41.935696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:42.011383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9649d-647f-41c2-adcd-1afe7b164bd2 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:42.047373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af50113d-0b21-4431-b4f4-04d883c22db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:42.048371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50113d-0b21-4431-b4f4-04d883c22db0 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:42.048777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50113d-0b21-4431-b4f4-04d883c22db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:42.049154 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af50113d-0b21-4431-b4f4-04d883c22db0 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:42.050069 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50113d-0b21-4431-b4f4-04d883c22db0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:42.050364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50113d-0b21-4431-b4f4-04d883c22db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:42.051759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af50113d-0b21-4431-b4f4-04d883c22db0 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:42.059382 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 587/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Thu Sep 8 15:56:41 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:44.163511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069d8f5-fcc9-4fc3-bc4b-9fd0dd7e745a None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:44.164416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069d8f5-fcc9-4fc3-bc4b-9fd0dd7e745a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:44.164416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1069d8f5-fcc9-4fc3-bc4b-9fd0dd7e745a None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:44.166533 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 580/1167] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:44.175232 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5f6dd-422f-446e-9ee2-bbeb2ffc78c6 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:44.175705 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5f6dd-422f-446e-9ee2-bbeb2ffc78c6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:44.175805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba5f6dd-422f-446e-9ee2-bbeb2ffc78c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:44.200460 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-cba5f6dd-422f-446e-9ee2-bbeb2ffc78c6 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:44.201637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-cba5f6dd-422f-446e-9ee2-bbeb2ffc78c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:44.272064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 588/1168] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:44.537524 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069d8f5-fcc9-4fc3-bc4b-9fd0dd7e745a None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:44.604559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:44.656785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1069d8f5-fcc9-4fc3-bc4b-9fd0dd7e745a None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:44.695525 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fda789-9778-4032-b769-cb9ce5cd9cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:44.696131 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fda789-9778-4032-b769-cb9ce5cd9cc8 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:44.696410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fda789-9778-4032-b769-cb9ce5cd9cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:44.696680 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fda789-9778-4032-b769-cb9ce5cd9cc8 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:44.697358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fda789-9778-4032-b769-cb9ce5cd9cc8 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:44.698773 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fda789-9778-4032-b769-cb9ce5cd9cc8 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:44.717828 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 581/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Thu Sep 8 15:56:44 2022] POST /identity/v3/domains => generated 267 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:56:46.850708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed26cb-caca-47ce-99d0-e744d5b2596b None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:46.851306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed26cb-caca-47ce-99d0-e744d5b2596b None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:46.851306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed26cb-caca-47ce-99d0-e744d5b2596b None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:46.853220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 589/1170] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:46.859357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c31c08-60f3-470c-bd68-643d0d6d2ca3 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:46.860109 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c31c08-60f3-470c-bd68-643d0d6d2ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:46.860234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c31c08-60f3-470c-bd68-643d0d6d2ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:46.881506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-12c31c08-60f3-470c-bd68-643d0d6d2ca3 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:46.882682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-12c31c08-60f3-470c-bd68-643d0d6d2ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:46.950888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 582/1171] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:47.223068 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed26cb-caca-47ce-99d0-e744d5b2596b None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:47.258669 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:47.339910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed26cb-caca-47ce-99d0-e744d5b2596b None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:47.377220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808ee1c-64dc-4bf3-b844-b592989ae5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:47.377867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808ee1c-64dc-4bf3-b844-b592989ae5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:47.378138 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808ee1c-64dc-4bf3-b844-b592989ae5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:47.378398 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808ee1c-64dc-4bf3-b844-b592989ae5c1 None admin] PATH_INFO: `/v3/domains/e2426fd3e8ef412aa7b0455177be28e2` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:47.389151 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808ee1c-64dc-4bf3-b844-b592989ae5c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=e2426fd3e8ef412aa7b0455177be28e2)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:47.390597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808ee1c-64dc-4bf3-b844-b592989ae5c1 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:47.392197 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 590/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Thu Sep 8 15:56:47 2022] GET /identity/v3/domains/e2426fd3e8ef412aa7b0455177be28e2 => generated 267 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:56:47.398344 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c31c08-60f3-470c-bd68-643d0d6d2ca3 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:47.439951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:47.501547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c31c08-60f3-470c-bd68-643d0d6d2ca3 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:47.546177 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c16a24-8251-4a5c-900a-663b70887b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:47.546780 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c16a24-8251-4a5c-900a-663b70887b21 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:47.547061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c16a24-8251-4a5c-900a-663b70887b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:47.547316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c16a24-8251-4a5c-900a-663b70887b21 None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:47.547994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c16a24-8251-4a5c-900a-663b70887b21 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:47.549477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c16a24-8251-4a5c-900a-663b70887b21 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:47.582192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 583/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Sep 8 15:56:47 2022] POST /identity/v3/projects => generated 361 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:56:49.773911 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0f8db-1ac0-408c-82a7-ca12e5dc4a0d None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:49.774827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0f8db-1ac0-408c-82a7-ca12e5dc4a0d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:49.774827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0f8db-1ac0-408c-82a7-ca12e5dc4a0d None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:49.776995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 591/1174] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:49.782535 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e0102-a0d2-4576-9e2a-b1eb07ca78aa None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:49.782978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e0102-a0d2-4576-9e2a-b1eb07ca78aa None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:49.783086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e0102-a0d2-4576-9e2a-b1eb07ca78aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:49.803358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-497e0102-a0d2-4576-9e2a-b1eb07ca78aa None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:49.804578 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-497e0102-a0d2-4576-9e2a-b1eb07ca78aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:49.873117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 584/1175] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:50.142553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0f8db-1ac0-408c-82a7-ca12e5dc4a0d None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:50.244527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:50.263619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0f8db-1ac0-408c-82a7-ca12e5dc4a0d None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:50.298748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0464c16d-e0e9-4c88-b117-1b9d3b79617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:50.299393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464c16d-e0e9-4c88-b117-1b9d3b79617f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:50.299795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464c16d-e0e9-4c88-b117-1b9d3b79617f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:50.300067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464c16d-e0e9-4c88-b117-1b9d3b79617f None admin] PATH_INFO: `/v3/domains/e2426fd3e8ef412aa7b0455177be28e2` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:50.301309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464c16d-e0e9-4c88-b117-1b9d3b79617f None admin] RBAC: Authorizing `identity:get_domain(domain_id=e2426fd3e8ef412aa7b0455177be28e2)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:50.302753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0464c16d-e0e9-4c88-b117-1b9d3b79617f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:50.304374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 592/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Thu Sep 8 15:56:50 2022] GET /identity/v3/domains/e2426fd3e8ef412aa7b0455177be28e2 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:50.311135 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e0102-a0d2-4576-9e2a-b1eb07ca78aa None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:50.340340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:50.421473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e0102-a0d2-4576-9e2a-b1eb07ca78aa None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:50.461121 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4515fb8-072e-4fff-8c0b-8964080c8903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:50.461717 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4515fb8-072e-4fff-8c0b-8964080c8903 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:50.461987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4515fb8-072e-4fff-8c0b-8964080c8903 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:50.462256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4515fb8-072e-4fff-8c0b-8964080c8903 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:50.462893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4515fb8-072e-4fff-8c0b-8964080c8903 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:50.464345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4515fb8-072e-4fff-8c0b-8964080c8903 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:50.488839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 585/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:56:50 2022] POST /identity/v3/users => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:56:52.634668 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fe88f-72c3-426a-afcb-b2cecd409061 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:52.635238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fe88f-72c3-426a-afcb-b2cecd409061 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:52.635238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fe88f-72c3-426a-afcb-b2cecd409061 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:52.636672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 593/1178] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:52.643314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f47f5-f129-4982-8c08-798d4ec8893b None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:52.643788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f47f5-f129-4982-8c08-798d4ec8893b None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:52.643880 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f47f5-f129-4982-8c08-798d4ec8893b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:52.663469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-e97f47f5-f129-4982-8c08-798d4ec8893b None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:52.664669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e97f47f5-f129-4982-8c08-798d4ec8893b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:52.732208 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 586/1179] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:52.995915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fe88f-72c3-426a-afcb-b2cecd409061 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:53.105882 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fe88f-72c3-426a-afcb-b2cecd409061 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:53.140751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f55279-eb81-45de-8881-ab72295f58e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:53.141363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f55279-eb81-45de-8881-ab72295f58e9 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:53.141631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f55279-eb81-45de-8881-ab72295f58e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:53.141899 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f55279-eb81-45de-8881-ab72295f58e9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:53.153233 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f55279-eb81-45de-8881-ab72295f58e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-20f55279-eb81-45de-8881-ab72295f58e9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:56:53.157980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:53.159114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:53.160163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:53.160163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:53.160163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:53.160163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:53.160163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:56:53.160163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:56:53.160163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:56:53.160163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:56:53.160163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:56:53.160163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:56:53.160163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:53.160163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:56:53.160163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 594/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:56:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:53.165755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f47f5-f129-4982-8c08-798d4ec8893b None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:53.176402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:53.269435 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f47f5-f129-4982-8c08-798d4ec8893b None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:53.304906 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf024f-1e49-46b0-8977-b261421d0163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:53.305518 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf024f-1e49-46b0-8977-b261421d0163 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:53.305792 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf024f-1e49-46b0-8977-b261421d0163 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:53.306063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbf024f-1e49-46b0-8977-b261421d0163 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:53.306727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf024f-1e49-46b0-8977-b261421d0163 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:53.307031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf024f-1e49-46b0-8977-b261421d0163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:53.308444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbf024f-1e49-46b0-8977-b261421d0163 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:53.317458 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 587/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:56:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:53.324474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f55279-eb81-45de-8881-ab72295f58e9 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:53.344337 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:53.444566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f55279-eb81-45de-8881-ab72295f58e9 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:53.480500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9b884-2892-4f94-9c94-df72c800b2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:53.481116 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9b884-2892-4f94-9c94-df72c800b2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:53.481395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9b884-2892-4f94-9c94-df72c800b2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:53.481667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9b884-2892-4f94-9c94-df72c800b2d8 None admin] PATH_INFO: `/v3/users/1804c2c4521e45a49e366d84242e3e89` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:53.495350 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9b884-2892-4f94-9c94-df72c800b2d8 None admin] RBAC: Authorizing `identity:get_user(user_id=1804c2c4521e45a49e366d84242e3e89)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:53.496840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9b884-2892-4f94-9c94-df72c800b2d8 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:53.498428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 595/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:56:53 2022] GET /identity/v3/users/1804c2c4521e45a49e366d84242e3e89 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:53.504449 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf024f-1e49-46b0-8977-b261421d0163 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:53.571448 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:53.609791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbf024f-1e49-46b0-8977-b261421d0163 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:53.649459 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf91b64-e037-46c2-b5f3-95a273d6284a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:53.650059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf91b64-e037-46c2-b5f3-95a273d6284a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:53.650319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf91b64-e037-46c2-b5f3-95a273d6284a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:53.650593 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf91b64-e037-46c2-b5f3-95a273d6284a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:53.720551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf91b64-e037-46c2-b5f3-95a273d6284a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:53.722054 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf91b64-e037-46c2-b5f3-95a273d6284a None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:53.786832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:53.824292 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 588/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:56:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:56:53.830357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9b884-2892-4f94-9c94-df72c800b2d8 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:53.910676 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:53.942421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9b884-2892-4f94-9c94-df72c800b2d8 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:53.978155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073d3b7-3198-4d12-b7b4-57cdbdde6920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:53.978767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073d3b7-3198-4d12-b7b4-57cdbdde6920 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:53.979037 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073d3b7-3198-4d12-b7b4-57cdbdde6920 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:53.979308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073d3b7-3198-4d12-b7b4-57cdbdde6920 None admin] PATH_INFO: `/v3/projects/e52794fa42fa43498276351d1351d65d` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:53.980488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073d3b7-3198-4d12-b7b4-57cdbdde6920 None admin] RBAC: Authorizing `identity:get_project(project_id=e52794fa42fa43498276351d1351d65d)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:53.981911 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073d3b7-3198-4d12-b7b4-57cdbdde6920 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:53.983509 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 596/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:56:53 2022] GET /identity/v3/projects/e52794fa42fa43498276351d1351d65d => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:53.991145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf91b64-e037-46c2-b5f3-95a273d6284a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:54.091301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf91b64-e037-46c2-b5f3-95a273d6284a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:54.126717 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92acc8-1c11-4fcc-914f-5ec1dc98fd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:54.127370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92acc8-1c11-4fcc-914f-5ec1dc98fd24 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:54.127720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92acc8-1c11-4fcc-914f-5ec1dc98fd24 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:54.127987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92acc8-1c11-4fcc-914f-5ec1dc98fd24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:54.128744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92acc8-1c11-4fcc-914f-5ec1dc98fd24 None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=e52794fa42fa43498276351d1351d65d, user.id=1804c2c4521e45a49e366d84242e3e89) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:54.129084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92acc8-1c11-4fcc-914f-5ec1dc98fd24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:54.130520 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92acc8-1c11-4fcc-914f-5ec1dc98fd24 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:54.137118 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 589/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:56:53 2022] GET /identity/v3/role_assignments?user.id=1804c2c4521e45a49e366d84242e3e89&scope.project.id=e52794fa42fa43498276351d1351d65d&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:56.221725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e37dc2-49be-4b5a-a336-523685f2afd4 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:56.222251 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e37dc2-49be-4b5a-a336-523685f2afd4 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:56.222251 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e37dc2-49be-4b5a-a336-523685f2afd4 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:56.223694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 597/1186] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:56.230023 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0786584-d7bf-42a7-976a-82c487cf49b4 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:56.230776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0786584-d7bf-42a7-976a-82c487cf49b4 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:56.230959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0786584-d7bf-42a7-976a-82c487cf49b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:56.252665 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-e0786584-d7bf-42a7-976a-82c487cf49b4 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:56.253868 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e0786584-d7bf-42a7-976a-82c487cf49b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:56.281856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:56.325758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 590/1187] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:56.604396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e37dc2-49be-4b5a-a336-523685f2afd4 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:56.718836 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e37dc2-49be-4b5a-a336-523685f2afd4 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:56.756649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cef4ec-6c8f-491e-802c-b8be50b1c9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:56.757278 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cef4ec-6c8f-491e-802c-b8be50b1c9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:56.757550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cef4ec-6c8f-491e-802c-b8be50b1c9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:56.757837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cef4ec-6c8f-491e-802c-b8be50b1c9c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:56.766110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:56.769909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cef4ec-6c8f-491e-802c-b8be50b1c9c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-b0cef4ec-6c8f-491e-802c-b8be50b1c9c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:56:56.779137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:56:56.780319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:56:56.781212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:56:56.781212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:56:56.781212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:56:56.781212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:56:56.781212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:56:56.781212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:56:56.781212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:56:56.781212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:56:56.781212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:56:56.781212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:56:56.781212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:56:56.781212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:56:56.781212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 598/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:56:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:56:56.786811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0786584-d7bf-42a7-976a-82c487cf49b4 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:56.886884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0786584-d7bf-42a7-976a-82c487cf49b4 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:56.922320 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344bb21-a999-44db-8724-b85b75bae3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:56.922913 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344bb21-a999-44db-8724-b85b75bae3cc None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:56.923183 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344bb21-a999-44db-8724-b85b75bae3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:56.923481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344bb21-a999-44db-8724-b85b75bae3cc None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:56.924127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344bb21-a999-44db-8724-b85b75bae3cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:56:56.924446 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344bb21-a999-44db-8724-b85b75bae3cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:56.925801 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344bb21-a999-44db-8724-b85b75bae3cc None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:56.933744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 591/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:56:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:56.940331 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cef4ec-6c8f-491e-802c-b8be50b1c9c9 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:57.064968 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cef4ec-6c8f-491e-802c-b8be50b1c9c9 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:57.104431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1133cbd-fddc-4368-9cfd-b867368f711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:57.105055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1133cbd-fddc-4368-9cfd-b867368f711f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:57.105340 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1133cbd-fddc-4368-9cfd-b867368f711f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:57.105602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1133cbd-fddc-4368-9cfd-b867368f711f None admin] PATH_INFO: `/v3/users/1804c2c4521e45a49e366d84242e3e89` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:57.106852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1133cbd-fddc-4368-9cfd-b867368f711f None admin] RBAC: Authorizing `identity:get_user(user_id=1804c2c4521e45a49e366d84242e3e89)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:57.108330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1133cbd-fddc-4368-9cfd-b867368f711f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:57.109923 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 599/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:56:56 2022] GET /identity/v3/users/1804c2c4521e45a49e366d84242e3e89 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:57.116288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344bb21-a999-44db-8724-b85b75bae3cc None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:57.142695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:57.227573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344bb21-a999-44db-8724-b85b75bae3cc None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:57.270490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13448cc8-c07b-495b-915d-fa0d3df6e245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:57.271096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13448cc8-c07b-495b-915d-fa0d3df6e245 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:57.271387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13448cc8-c07b-495b-915d-fa0d3df6e245 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:57.271753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13448cc8-c07b-495b-915d-fa0d3df6e245 None admin] PATH_INFO: `/v3/projects/e52794fa42fa43498276351d1351d65d` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:57.272957 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13448cc8-c07b-495b-915d-fa0d3df6e245 None admin] RBAC: Authorizing `identity:get_project(project_id=e52794fa42fa43498276351d1351d65d)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:57.274404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13448cc8-c07b-495b-915d-fa0d3df6e245 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:57.276190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 592/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:56:57 2022] GET /identity/v3/projects/e52794fa42fa43498276351d1351d65d => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:56:57.282553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1133cbd-fddc-4368-9cfd-b867368f711f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:56:57.353935 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:56:57.391761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1133cbd-fddc-4368-9cfd-b867368f711f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:56:57.426982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2b25e2-b4d7-4a8c-b078-97b35eac5b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:56:57.427585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b25e2-b4d7-4a8c-b078-97b35eac5b67 None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:57.427857 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b25e2-b4d7-4a8c-b078-97b35eac5b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:57.428123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2b25e2-b4d7-4a8c-b078-97b35eac5b67 None admin] PATH_INFO: `/v3/projects/e52794fa42fa43498276351d1351d65d/users/1804c2c4521e45a49e366d84242e3e89/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:57.430107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b25e2-b4d7-4a8c-b078-97b35eac5b67 None admin] RBAC: Authorizing `identity:create_grant(project_id=e52794fa42fa43498276351d1351d65d, user_id=1804c2c4521e45a49e366d84242e3e89, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:56:57.431560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2b25e2-b4d7-4a8c-b078-97b35eac5b67 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:56:57.446840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 600/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:56:57 2022] PUT /identity/v3/projects/e52794fa42fa43498276351d1351d65d/users/1804c2c4521e45a49e366d84242e3e89/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:56:59.551058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b567b-05de-47a9-8084-bfef2def2533 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:59.552032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b567b-05de-47a9-8084-bfef2def2533 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:59.552032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b567b-05de-47a9-8084-bfef2def2533 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:59.554123 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 593/1193] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:56:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:56:59.560625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bad0df-5fdf-4fab-a3c2-2d134f5cf993 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:56:59.561544 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bad0df-5fdf-4fab-a3c2-2d134f5cf993 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:56:59.561544 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bad0df-5fdf-4fab-a3c2-2d134f5cf993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:56:59.587959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-00bad0df-5fdf-4fab-a3c2-2d134f5cf993 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:56:59.589143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-00bad0df-5fdf-4fab-a3c2-2d134f5cf993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:56:59.660971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 601/1194] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:56:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:56:59.935623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b567b-05de-47a9-8084-bfef2def2533 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:57:00.036400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:57:00.047222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b567b-05de-47a9-8084-bfef2def2533 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:57:00.081080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52664b37-7d4e-415b-bcc8-04ec6ae9d32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:57:00.081696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52664b37-7d4e-415b-bcc8-04ec6ae9d32f None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:57:00.081963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52664b37-7d4e-415b-bcc8-04ec6ae9d32f None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:57:00.082208 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52664b37-7d4e-415b-bcc8-04ec6ae9d32f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:57:00.093033 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52664b37-7d4e-415b-bcc8-04ec6ae9d32f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-52664b37-7d4e-415b-bcc8-04ec6ae9d32f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:57:00.097806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:57:00.098818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:57:00.100199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:57:00.100199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:57:00.100199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:57:00.100199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:57:00.100199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:57:00.100199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:57:00.100199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:57:00.100199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:57:00.100199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:57:00.100199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:57:00.100199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:57:00.100199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:57:00.100199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 594/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:56:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:57:00.107329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bad0df-5fdf-4fab-a3c2-2d134f5cf993 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:57:00.175099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:57:00.222926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bad0df-5fdf-4fab-a3c2-2d134f5cf993 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:57:00.257076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58d2a3-8bf0-4bfb-a4d1-16be5cb91e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:57:00.257672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58d2a3-8bf0-4bfb-a4d1-16be5cb91e90 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:57:00.257946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58d2a3-8bf0-4bfb-a4d1-16be5cb91e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:57:00.258211 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58d2a3-8bf0-4bfb-a4d1-16be5cb91e90 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:57:00.258867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58d2a3-8bf0-4bfb-a4d1-16be5cb91e90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:57:00.259163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58d2a3-8bf0-4bfb-a4d1-16be5cb91e90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:57:00.260566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58d2a3-8bf0-4bfb-a4d1-16be5cb91e90 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:57:00.268602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 602/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:57:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:57:00.278057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52664b37-7d4e-415b-bcc8-04ec6ae9d32f None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:57:00.389912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52664b37-7d4e-415b-bcc8-04ec6ae9d32f None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:57:00.424514 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121aa5ae-3375-437c-a0c1-ef5dca556eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:57:00.425139 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121aa5ae-3375-437c-a0c1-ef5dca556eca None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:57:00.425417 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121aa5ae-3375-437c-a0c1-ef5dca556eca None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:57:00.425688 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121aa5ae-3375-437c-a0c1-ef5dca556eca None admin] PATH_INFO: `/v3/users/1804c2c4521e45a49e366d84242e3e89` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:57:00.426858 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121aa5ae-3375-437c-a0c1-ef5dca556eca None admin] RBAC: Authorizing `identity:get_user(user_id=1804c2c4521e45a49e366d84242e3e89)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:57:00.428334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121aa5ae-3375-437c-a0c1-ef5dca556eca None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:57:00.429914 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 595/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Thu Sep 8 15:57:00 2022] GET /identity/v3/users/1804c2c4521e45a49e366d84242e3e89 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:57:00.436834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58d2a3-8bf0-4bfb-a4d1-16be5cb91e90 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:57:00.543268 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58d2a3-8bf0-4bfb-a4d1-16be5cb91e90 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:57:00.563210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:57:00.579331 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e391bc9-4a48-4f49-8ca0-6aaf9efdbf06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:57:00.579959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e391bc9-4a48-4f49-8ca0-6aaf9efdbf06 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:57:00.580233 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e391bc9-4a48-4f49-8ca0-6aaf9efdbf06 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:57:00.580480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e391bc9-4a48-4f49-8ca0-6aaf9efdbf06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:57:00.654329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e391bc9-4a48-4f49-8ca0-6aaf9efdbf06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:57:00.655805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e391bc9-4a48-4f49-8ca0-6aaf9efdbf06 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:57:00.758661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 603/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:57:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:57:00.765213 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121aa5ae-3375-437c-a0c1-ef5dca556eca None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:57:00.787977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:57:00.869268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121aa5ae-3375-437c-a0c1-ef5dca556eca None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:57:00.903906 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e07fa-8a82-487c-967f-89537e15754b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:57:00.904522 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e07fa-8a82-487c-967f-89537e15754b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:57:00.904796 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e07fa-8a82-487c-967f-89537e15754b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:57:00.905063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e07fa-8a82-487c-967f-89537e15754b None admin] PATH_INFO: `/v3/projects/e52794fa42fa43498276351d1351d65d` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:57:00.906233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e07fa-8a82-487c-967f-89537e15754b None admin] RBAC: Authorizing `identity:get_project(project_id=e52794fa42fa43498276351d1351d65d)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:57:00.907692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e07fa-8a82-487c-967f-89537e15754b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:57:00.909349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 596/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Thu Sep 8 15:57:00 2022] GET /identity/v3/projects/e52794fa42fa43498276351d1351d65d => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:57:00.915824 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e391bc9-4a48-4f49-8ca0-6aaf9efdbf06 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:57:00.937434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:57:01.022166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e391bc9-4a48-4f49-8ca0-6aaf9efdbf06 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:57:01.058115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbebde7-d06d-45df-86dc-4efac8355e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:57:01.058704 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbebde7-d06d-45df-86dc-4efac8355e92 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:57:01.058978 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbebde7-d06d-45df-86dc-4efac8355e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:57:01.059245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbebde7-d06d-45df-86dc-4efac8355e92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:57:01.059998 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbebde7-d06d-45df-86dc-4efac8355e92 None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=e52794fa42fa43498276351d1351d65d, user.id=1804c2c4521e45a49e366d84242e3e89) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:57:01.060300 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbebde7-d06d-45df-86dc-4efac8355e92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:57:01.061733 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbebde7-d06d-45df-86dc-4efac8355e92 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:57:01.067684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 604/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:57:00 2022] GET /identity/v3/role_assignments?user.id=1804c2c4521e45a49e366d84242e3e89&scope.project.id=e52794fa42fa43498276351d1351d65d&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:57:10.898649 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=56 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:57:11.064886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=58 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:57:57.146631 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad563e9c-a460-4400-9c2f-faaa2cd14d1a None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:57:57.147249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad563e9c-a460-4400-9c2f-faaa2cd14d1a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:57:57.147249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad563e9c-a460-4400-9c2f-faaa2cd14d1a None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:57:57.148908 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 597/1201] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 15:57:57.154756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247168f4-fb9b-4853-874e-b45d48773953 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:57:57.155389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247168f4-fb9b-4853-874e-b45d48773953 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:57:57.155599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247168f4-fb9b-4853-874e-b45d48773953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:57:57.175827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-247168f4-fb9b-4853-874e-b45d48773953 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:57:57.177041 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-247168f4-fb9b-4853-874e-b45d48773953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:57:57.252873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 605/1202] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:57:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:57:57.531284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad563e9c-a460-4400-9c2f-faaa2cd14d1a None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:57:57.655099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad563e9c-a460-4400-9c2f-faaa2cd14d1a None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:57:57.701902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de2f1f-afcc-40a1-a443-06986575b4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:57:57.702567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de2f1f-afcc-40a1-a443-06986575b4c9 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:57:57.702840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de2f1f-afcc-40a1-a443-06986575b4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:57:57.703103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3de2f1f-afcc-40a1-a443-06986575b4c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:57:57.716236 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de2f1f-afcc-40a1-a443-06986575b4c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:57:57.717791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3de2f1f-afcc-40a1-a443-06986575b4c9 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-f3de2f1f-afcc-40a1-a443-06986575b4c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:57:57.735370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:57:57.736586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:57:57.742696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:57:57.742696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:57:57.742696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:57:57.742696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 598/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:57:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:57:57.743023 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247168f4-fb9b-4853-874e-b45d48773953 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:57:57.864874 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247168f4-fb9b-4853-874e-b45d48773953 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:57:57.901343 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:57:57.915980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b883d93b-72b1-40fc-9f29-a502d05d00c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:57:57.917048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883d93b-72b1-40fc-9f29-a502d05d00c9 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:57:57.917421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883d93b-72b1-40fc-9f29-a502d05d00c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:57:57.917828 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b883d93b-72b1-40fc-9f29-a502d05d00c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:57:57.919185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883d93b-72b1-40fc-9f29-a502d05d00c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:57:57.919569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883d93b-72b1-40fc-9f29-a502d05d00c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:57:57.922413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b883d93b-72b1-40fc-9f29-a502d05d00c9 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:57:57.951505 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 606/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:57:57.952554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de2f1f-afcc-40a1-a443-06986575b4c9 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:57:58.051183 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:57:58.143304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3de2f1f-afcc-40a1-a443-06986575b4c9 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:57:58.200710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff307007-bbf8-4200-aa12-3824d7c9b567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:57:58.201477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff307007-bbf8-4200-aa12-3824d7c9b567 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:57:58.201919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff307007-bbf8-4200-aa12-3824d7c9b567 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:57:58.202134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff307007-bbf8-4200-aa12-3824d7c9b567 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:57:58.202926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff307007-bbf8-4200-aa12-3824d7c9b567 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:57:58.204644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff307007-bbf8-4200-aa12-3824d7c9b567 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:57:58.230374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 599/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Thu Sep 8 15:57:57 2022] POST /identity/v3/users => generated 251 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:58:00.628275 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fc774-791f-45c3-9089-46f2a8721188 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:00.629176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fc774-791f-45c3-9089-46f2a8721188 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:00.629176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fc774-791f-45c3-9089-46f2a8721188 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:00.630540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 607/1206] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:58:00.636387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab57fd-b6e4-49d8-805a-ccade8c69dbb None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:00.636886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab57fd-b6e4-49d8-805a-ccade8c69dbb None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:00.636986 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab57fd-b6e4-49d8-805a-ccade8c69dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:00.656596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-72ab57fd-b6e4-49d8-805a-ccade8c69dbb None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:58:00.658013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-72ab57fd-b6e4-49d8-805a-ccade8c69dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:58:00.732093 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 600/1207] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:58:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:58:01.026089 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fc774-791f-45c3-9089-46f2a8721188 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:01.141363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fc774-791f-45c3-9089-46f2a8721188 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:01.189107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeec15b-84f7-4c70-a536-9be7e7e7ccb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:01.189904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeec15b-84f7-4c70-a536-9be7e7e7ccb1 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:01.190212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeec15b-84f7-4c70-a536-9be7e7e7ccb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:01.190511 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeec15b-84f7-4c70-a536-9be7e7e7ccb1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:01.203569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeec15b-84f7-4c70-a536-9be7e7e7ccb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-adeec15b-84f7-4c70-a536-9be7e7e7ccb1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:58:01.209142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:01.211975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:01.213088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:01.213088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:01.213088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:01.213088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:01.213088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:58:01.213088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:58:01.213088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:58:01.213088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:58:01.213088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:58:01.213088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:58:01.213088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:58:01.213088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:01.213088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 608/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:58:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:01.217556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab57fd-b6e4-49d8-805a-ccade8c69dbb None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:01.263251 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:01.335199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab57fd-b6e4-49d8-805a-ccade8c69dbb None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:01.376022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39f812-103e-430a-b711-9fa50c0a0597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:01.376787 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39f812-103e-430a-b711-9fa50c0a0597 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:01.377115 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39f812-103e-430a-b711-9fa50c0a0597 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:01.377444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39f812-103e-430a-b711-9fa50c0a0597 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:01.378343 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39f812-103e-430a-b711-9fa50c0a0597 None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:01.378765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39f812-103e-430a-b711-9fa50c0a0597 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:01.380281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39f812-103e-430a-b711-9fa50c0a0597 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:01.390606 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 601/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:58:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:58:01.397406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeec15b-84f7-4c70-a536-9be7e7e7ccb1 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:01.458059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:01.517469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeec15b-84f7-4c70-a536-9be7e7e7ccb1 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:01.556588 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ed8a3-8ee2-409d-9b30-944b68dd0deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:01.558830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ed8a3-8ee2-409d-9b30-944b68dd0deb None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:01.558830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ed8a3-8ee2-409d-9b30-944b68dd0deb None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:01.558830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ed8a3-8ee2-409d-9b30-944b68dd0deb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:01.569976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ed8a3-8ee2-409d-9b30-944b68dd0deb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:01.571523 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ed8a3-8ee2-409d-9b30-944b68dd0deb None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-571ed8a3-8ee2-409d-9b30-944b68dd0deb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:58:01.590518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:01.592107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:58:01.593052 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:58:01.593052 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:01.593052 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:01.593052 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 609/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:01.597725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39f812-103e-430a-b711-9fa50c0a0597 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:01.705008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:01.708424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39f812-103e-430a-b711-9fa50c0a0597 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:01.750699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078acd9-f98e-4d1c-a205-4acc01f606f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:01.751400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078acd9-f98e-4d1c-a205-4acc01f606f2 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:01.751822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078acd9-f98e-4d1c-a205-4acc01f606f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:01.752033 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078acd9-f98e-4d1c-a205-4acc01f606f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:01.752784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078acd9-f98e-4d1c-a205-4acc01f606f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:01.753105 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078acd9-f98e-4d1c-a205-4acc01f606f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:01.754601 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a078acd9-f98e-4d1c-a205-4acc01f606f2 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:01.769198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 602/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:58:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:01.774720 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ed8a3-8ee2-409d-9b30-944b68dd0deb None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:01.884767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ed8a3-8ee2-409d-9b30-944b68dd0deb None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:01.885663 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:01.920076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed10af7-7003-455b-9205-d09328b69558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:01.920714 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed10af7-7003-455b-9205-d09328b69558 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:01.920990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed10af7-7003-455b-9205-d09328b69558 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:01.921258 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed10af7-7003-455b-9205-d09328b69558 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:01.929206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed10af7-7003-455b-9205-d09328b69558 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:01.930956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed10af7-7003-455b-9205-d09328b69558 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-5ed10af7-7003-455b-9205-d09328b69558 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:58:01.944036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:01.945064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:01.946683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:01.946683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:01.946683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:58:01.946683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:58:01.946683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:58:01.946683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:58:01.946683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:58:01.946683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:58:01.946683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 15:58:01.946683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:01.946683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 610/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:01.951115 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078acd9-f98e-4d1c-a205-4acc01f606f2 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:02.075839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078acd9-f98e-4d1c-a205-4acc01f606f2 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:02.113534 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7afc9-6293-40ab-9f8c-a0fd6c93d1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:02.114168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7afc9-6293-40ab-9f8c-a0fd6c93d1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:02.114553 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7afc9-6293-40ab-9f8c-a0fd6c93d1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:02.114753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d7afc9-6293-40ab-9f8c-a0fd6c93d1c4 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:02.115598 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7afc9-6293-40ab-9f8c-a0fd6c93d1c4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:02.115894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7afc9-6293-40ab-9f8c-a0fd6c93d1c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:02.117426 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d7afc9-6293-40ab-9f8c-a0fd6c93d1c4 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:02.137980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 603/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Thu Sep 8 15:58:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:02.143780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed10af7-7003-455b-9205-d09328b69558 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:02.262744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed10af7-7003-455b-9205-d09328b69558 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:02.301485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130be89-8580-49f5-b823-f9b23a9e1f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:02.302160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130be89-8580-49f5-b823-f9b23a9e1f24 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:02.302459 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130be89-8580-49f5-b823-f9b23a9e1f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:02.302743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130be89-8580-49f5-b823-f9b23a9e1f24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:02.335248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:02.377116 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130be89-8580-49f5-b823-f9b23a9e1f24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:02.378623 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130be89-8580-49f5-b823-f9b23a9e1f24 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:02.506700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 611/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:58:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:58:02.508106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7afc9-6293-40ab-9f8c-a0fd6c93d1c4 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:02.531924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:02.628363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d7afc9-6293-40ab-9f8c-a0fd6c93d1c4 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:02.663370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199cb00-98ae-4d23-947f-fe521f484c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:02.664037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199cb00-98ae-4d23-947f-fe521f484c93 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:02.664329 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199cb00-98ae-4d23-947f-fe521f484c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:02.664611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199cb00-98ae-4d23-947f-fe521f484c93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:02.675006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199cb00-98ae-4d23-947f-fe521f484c93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:02.676681 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199cb00-98ae-4d23-947f-fe521f484c93 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-f199cb00-98ae-4d23-947f-fe521f484c93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:58:02.691827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:02.693447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:58:02.694821 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:58:02.694821 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:02.694821 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:58:02.694821 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 604/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:02.698897 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130be89-8580-49f5-b823-f9b23a9e1f24 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:02.762546 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:02.802901 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130be89-8580-49f5-b823-f9b23a9e1f24 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:02.841188 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40bf83-f5e7-48ee-87aa-26120961a2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:02.843547 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40bf83-f5e7-48ee-87aa-26120961a2ef None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:02.843865 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40bf83-f5e7-48ee-87aa-26120961a2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:02.844185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40bf83-f5e7-48ee-87aa-26120961a2ef None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:02.844872 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40bf83-f5e7-48ee-87aa-26120961a2ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:02.845306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40bf83-f5e7-48ee-87aa-26120961a2ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:02.846695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c40bf83-f5e7-48ee-87aa-26120961a2ef None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:02.863475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 612/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:02.865696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199cb00-98ae-4d23-947f-fe521f484c93 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:02.943246 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:02.978400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199cb00-98ae-4d23-947f-fe521f484c93 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:03.018711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ced89a-8636-42ed-a426-2c3ec2de22c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:03.019277 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ced89a-8636-42ed-a426-2c3ec2de22c7 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:03.019580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ced89a-8636-42ed-a426-2c3ec2de22c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:03.019846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ced89a-8636-42ed-a426-2c3ec2de22c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:03.027574 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ced89a-8636-42ed-a426-2c3ec2de22c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:03.029295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ced89a-8636-42ed-a426-2c3ec2de22c7 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:03.040813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-55ced89a-8636-42ed-a426-2c3ec2de22c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:58:03.040813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:03.040813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:03.040813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:03.040813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:03.040813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:03.040813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:03.040813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:03.040813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:03.040813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:03.040813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:03.040813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:03.040813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:58:03.040813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:58:03.040813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:03.041778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:03.042716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:58:03.042716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:58:03.042716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:58:03.042716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:58:03.042716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:58:03.042716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:58:03.042716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:58:03.042716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:58:03.042716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 605/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:58:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:03.048322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40bf83-f5e7-48ee-87aa-26120961a2ef None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:03.153280 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40bf83-f5e7-48ee-87aa-26120961a2ef None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:03.181478 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:03.199877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bba341-8fb9-4b80-bc5a-d07fca361778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:03.200556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bba341-8fb9-4b80-bc5a-d07fca361778 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:03.200805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bba341-8fb9-4b80-bc5a-d07fca361778 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:03.201075 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bba341-8fb9-4b80-bc5a-d07fca361778 None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:03.201769 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bba341-8fb9-4b80-bc5a-d07fca361778 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:03.202073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bba341-8fb9-4b80-bc5a-d07fca361778 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:03.203588 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bba341-8fb9-4b80-bc5a-d07fca361778 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:03.216198 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 613/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:58:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:03.222608 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ced89a-8636-42ed-a426-2c3ec2de22c7 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:03.330725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ced89a-8636-42ed-a426-2c3ec2de22c7 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:03.359820 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:03.366118 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c426ee6-5103-4fd4-a19e-6a6e3ce8d2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:03.366732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c426ee6-5103-4fd4-a19e-6a6e3ce8d2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:03.367047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c426ee6-5103-4fd4-a19e-6a6e3ce8d2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:03.369286 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c426ee6-5103-4fd4-a19e-6a6e3ce8d2c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:03.369286 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c426ee6-5103-4fd4-a19e-6a6e3ce8d2c5 None admin] RBAC: Adding query filter params (role.id=b256252b45e541a491376a134f502183, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=dd28bb31b55e45668212c9ab69ed3ac3) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:03.369286 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c426ee6-5103-4fd4-a19e-6a6e3ce8d2c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:03.369820 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c426ee6-5103-4fd4-a19e-6a6e3ce8d2c5 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:03.376125 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 606/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:58:03 2022] GET /identity/v3/role_assignments?user.id=dd28bb31b55e45668212c9ab69ed3ac3&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=b256252b45e541a491376a134f502183 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:05.608562 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19f65b-80dd-4304-9ca1-71fba8ad656d None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:05.609137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19f65b-80dd-4304-9ca1-71fba8ad656d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:05.609279 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19f65b-80dd-4304-9ca1-71fba8ad656d None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:05.611402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 614/1220] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:58:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:58:05.617704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd6459-7bb9-44e8-acdb-ade26fbf117a None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:05.618136 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd6459-7bb9-44e8-acdb-ade26fbf117a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:05.618253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd6459-7bb9-44e8-acdb-ade26fbf117a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:05.638253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-4dbd6459-7bb9-44e8-acdb-ade26fbf117a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:58:05.639482 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4dbd6459-7bb9-44e8-acdb-ade26fbf117a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:58:05.718334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 607/1221] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:58:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:58:06.036748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff19f65b-80dd-4304-9ca1-71fba8ad656d None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:06.147453 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff19f65b-80dd-4304-9ca1-71fba8ad656d None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:06.185301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bb985-7ef9-42c5-a53e-a74e4b290e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:06.185995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bb985-7ef9-42c5-a53e-a74e4b290e07 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:06.186309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bb985-7ef9-42c5-a53e-a74e4b290e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:06.186629 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bb985-7ef9-42c5-a53e-a74e4b290e07 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:06.199073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794bb985-7ef9-42c5-a53e-a74e4b290e07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-794bb985-7ef9-42c5-a53e-a74e4b290e07 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:58:06.204497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:06.205762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:06.207407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:06.207407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:06.207407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:06.207407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:06.207407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:58:06.207407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:58:06.207407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:58:06.207407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:58:06.207407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:58:06.207407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:58:06.207407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:58:06.207407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:06.207407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 615/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:58:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:06.212530 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd6459-7bb9-44e8-acdb-ade26fbf117a None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:06.329351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd6459-7bb9-44e8-acdb-ade26fbf117a None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:06.355278 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:06.366212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036d0e6-5497-492a-ad74-a6f84dcdd8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:06.366807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036d0e6-5497-492a-ad74-a6f84dcdd8f4 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:06.367101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036d0e6-5497-492a-ad74-a6f84dcdd8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:06.367344 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8036d0e6-5497-492a-ad74-a6f84dcdd8f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:06.368044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036d0e6-5497-492a-ad74-a6f84dcdd8f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:06.368340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036d0e6-5497-492a-ad74-a6f84dcdd8f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:06.369717 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8036d0e6-5497-492a-ad74-a6f84dcdd8f4 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:06.383954 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 608/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:58:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:06.385260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bb985-7ef9-42c5-a53e-a74e4b290e07 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:06.430597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:06.489195 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bb985-7ef9-42c5-a53e-a74e4b290e07 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:06.524256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013097b-5234-43e4-bb69-b1c3be27f429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:06.524966 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013097b-5234-43e4-bb69-b1c3be27f429 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:06.525304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013097b-5234-43e4-bb69-b1c3be27f429 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:06.525578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013097b-5234-43e4-bb69-b1c3be27f429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:06.537298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013097b-5234-43e4-bb69-b1c3be27f429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:06.539771 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013097b-5234-43e4-bb69-b1c3be27f429 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-1013097b-5234-43e4-bb69-b1c3be27f429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:58:06.557079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:06.558367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:58:06.559475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:58:06.559475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:06.559475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:06.559475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 616/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:06.564513 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036d0e6-5497-492a-ad74-a6f84dcdd8f4 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:06.702286 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8036d0e6-5497-492a-ad74-a6f84dcdd8f4 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:06.738859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd83bf-20c9-4f32-9bbc-40e161ffafab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:06.739504 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd83bf-20c9-4f32-9bbc-40e161ffafab None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:06.739845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd83bf-20c9-4f32-9bbc-40e161ffafab None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:06.740111 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babd83bf-20c9-4f32-9bbc-40e161ffafab None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:06.740762 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd83bf-20c9-4f32-9bbc-40e161ffafab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:06.741062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd83bf-20c9-4f32-9bbc-40e161ffafab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:06.742769 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babd83bf-20c9-4f32-9bbc-40e161ffafab None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:06.757102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 609/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:06.763599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013097b-5234-43e4-bb69-b1c3be27f429 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:06.871562 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:06.879842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013097b-5234-43e4-bb69-b1c3be27f429 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:06.920886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e478fe-8587-4f03-9729-f99110caf31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:06.921527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e478fe-8587-4f03-9729-f99110caf31c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:06.921800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e478fe-8587-4f03-9729-f99110caf31c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:06.922070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e478fe-8587-4f03-9729-f99110caf31c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:06.929761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e478fe-8587-4f03-9729-f99110caf31c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:06.931229 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e478fe-8587-4f03-9729-f99110caf31c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-e3e478fe-8587-4f03-9729-f99110caf31c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:58:06.943990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:06.945159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:06.946432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:06.946432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:06.946432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:58:06.946432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:58:06.946432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:58:06.946432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:58:06.946432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:58:06.946432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:58:06.946432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 15:58:06.946432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:06.946432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 617/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:06.951280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd83bf-20c9-4f32-9bbc-40e161ffafab None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:06.999800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:07.057319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babd83bf-20c9-4f32-9bbc-40e161ffafab None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:07.094372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66cb93-fffd-4f13-8e61-9c6153b427f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:07.094997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66cb93-fffd-4f13-8e61-9c6153b427f4 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:07.095269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66cb93-fffd-4f13-8e61-9c6153b427f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:07.095565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66cb93-fffd-4f13-8e61-9c6153b427f4 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:07.097196 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66cb93-fffd-4f13-8e61-9c6153b427f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:07.097196 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66cb93-fffd-4f13-8e61-9c6153b427f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:07.098068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd66cb93-fffd-4f13-8e61-9c6153b427f4 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:07.121263 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 610/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Thu Sep 8 15:58:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:07.122953 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e478fe-8587-4f03-9729-f99110caf31c None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:07.242196 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e478fe-8587-4f03-9729-f99110caf31c None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:07.278240 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe0d78-7e90-4d6f-9c9f-5c8b1d93436f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:07.278880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe0d78-7e90-4d6f-9c9f-5c8b1d93436f None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:07.279156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe0d78-7e90-4d6f-9c9f-5c8b1d93436f None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:07.279426 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe0d78-7e90-4d6f-9c9f-5c8b1d93436f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:07.290861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe0d78-7e90-4d6f-9c9f-5c8b1d93436f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:07.292887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe0d78-7e90-4d6f-9c9f-5c8b1d93436f None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-f4fe0d78-7e90-4d6f-9c9f-5c8b1d93436f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:58:07.308422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:07.309757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:58:07.310932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:58:07.310932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:07.310932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:07.310932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 618/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:07.315984 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66cb93-fffd-4f13-8e61-9c6153b427f4 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:07.424867 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd66cb93-fffd-4f13-8e61-9c6153b427f4 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:07.427731 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:07.462394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366b059-ee74-470b-bece-339a5f713dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:07.463036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366b059-ee74-470b-bece-339a5f713dcc None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:07.463337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366b059-ee74-470b-bece-339a5f713dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:07.463655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366b059-ee74-470b-bece-339a5f713dcc None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:07.464402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366b059-ee74-470b-bece-339a5f713dcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:07.464770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366b059-ee74-470b-bece-339a5f713dcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:07.466290 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a366b059-ee74-470b-bece-339a5f713dcc None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:07.478693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 611/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:07.484634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe0d78-7e90-4d6f-9c9f-5c8b1d93436f None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:07.494948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:07.616011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe0d78-7e90-4d6f-9c9f-5c8b1d93436f None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:07.650660 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fea58-8d59-46df-9aa5-6b9294293abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:07.651288 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fea58-8d59-46df-9aa5-6b9294293abf None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:07.651591 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fea58-8d59-46df-9aa5-6b9294293abf None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:07.651857 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fea58-8d59-46df-9aa5-6b9294293abf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:07.658386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fea58-8d59-46df-9aa5-6b9294293abf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:07.659863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fea58-8d59-46df-9aa5-6b9294293abf None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:07.671358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-753fea58-8d59-46df-9aa5-6b9294293abf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:58:07.671358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:07.671358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:07.671358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:07.671358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:07.671358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:07.671358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:07.671358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:07.671358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:07.671358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:07.671358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:07.671358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:07.671358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:58:07.671358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:58:07.671358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:07.672734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:07.673817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:58:07.673817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:58:07.673817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:58:07.673817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:58:07.673817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:58:07.673817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:58:07.673817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:58:07.673817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:07.673817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 619/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:58:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:07.678427 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366b059-ee74-470b-bece-339a5f713dcc None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:07.784923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366b059-ee74-470b-bece-339a5f713dcc None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:07.819668 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcd969-2db7-4f62-8bf3-c53d4e7b685d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:07.820270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcd969-2db7-4f62-8bf3-c53d4e7b685d None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:07.820538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcd969-2db7-4f62-8bf3-c53d4e7b685d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:07.820800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcd969-2db7-4f62-8bf3-c53d4e7b685d None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:07.821487 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcd969-2db7-4f62-8bf3-c53d4e7b685d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:07.821781 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcd969-2db7-4f62-8bf3-c53d4e7b685d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:07.823202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbcd969-2db7-4f62-8bf3-c53d4e7b685d None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:07.835687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 612/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:58:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:07.842390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fea58-8d59-46df-9aa5-6b9294293abf None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:07.916194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:07.964605 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fea58-8d59-46df-9aa5-6b9294293abf None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:08.001503 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cb47d-c039-445b-91ad-b836543811c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:08.002169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cb47d-c039-445b-91ad-b836543811c3 None admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:08.002437 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cb47d-c039-445b-91ad-b836543811c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:08.002748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cb47d-c039-445b-91ad-b836543811c3 None admin] PATH_INFO: `/v3/projects/a825a0b1238d4382995da440944f453b/users/dd28bb31b55e45668212c9ab69ed3ac3/roles/b256252b45e541a491376a134f502183` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:08.018474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8cb47d-c039-445b-91ad-b836543811c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a825a0b1238d4382995da440944f453b, user_id=dd28bb31b55e45668212c9ab69ed3ac3, role_id=b256252b45e541a491376a134f502183)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:08.020536 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8cb47d-c039-445b-91ad-b836543811c3 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:08.034488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 620/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:58:07 2022] PUT /identity/v3/projects/a825a0b1238d4382995da440944f453b/users/dd28bb31b55e45668212c9ab69ed3ac3/roles/b256252b45e541a491376a134f502183 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:58:10.155019 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab2b98-5616-4ad6-9559-cdd81648cd35 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:10.155613 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab2b98-5616-4ad6-9559-cdd81648cd35 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:10.155613 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab2b98-5616-4ad6-9559-cdd81648cd35 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:10.157161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 613/1233] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:58:10.162764 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1097a-a54f-412d-b8f9-c356abaac5ff None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:10.163167 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1097a-a54f-412d-b8f9-c356abaac5ff None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:10.163350 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1097a-a54f-412d-b8f9-c356abaac5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:10.185447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-c5e1097a-a54f-412d-b8f9-c356abaac5ff None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:58:10.186781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c5e1097a-a54f-412d-b8f9-c356abaac5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:58:10.264747 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 621/1234] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:58:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:58:10.548029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab2b98-5616-4ad6-9559-cdd81648cd35 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:10.582127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:10.683134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab2b98-5616-4ad6-9559-cdd81648cd35 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:10.721336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59f8df-9991-4b17-a37a-d7aa8000d594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:10.722077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59f8df-9991-4b17-a37a-d7aa8000d594 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:10.722426 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59f8df-9991-4b17-a37a-d7aa8000d594 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:10.722699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59f8df-9991-4b17-a37a-d7aa8000d594 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:10.733445 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59f8df-9991-4b17-a37a-d7aa8000d594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-5c59f8df-9991-4b17-a37a-d7aa8000d594 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:58:10.738512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:10.739683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:10.744602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:10.744602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:10.744602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:10.744602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:10.744602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:58:10.744602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:58:10.744602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:58:10.744602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:58:10.744602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:58:10.744602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:58:10.744602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 15:58:10.744602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:58:10.744602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 614/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Thu Sep 8 15:58:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:10.747957 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1097a-a54f-412d-b8f9-c356abaac5ff None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:10.797700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:10.859390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1097a-a54f-412d-b8f9-c356abaac5ff None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:10.901742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1650f-49dc-4936-9ad1-e6bcb746a8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:10.902365 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1650f-49dc-4936-9ad1-e6bcb746a8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:10.902636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1650f-49dc-4936-9ad1-e6bcb746a8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:10.902903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e1650f-49dc-4936-9ad1-e6bcb746a8d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:10.903610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e1650f-49dc-4936-9ad1-e6bcb746a8d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:10.903910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e1650f-49dc-4936-9ad1-e6bcb746a8d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:10.905312 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e1650f-49dc-4936-9ad1-e6bcb746a8d2 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:10.914034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 622/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Thu Sep 8 15:58:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:10.920155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59f8df-9991-4b17-a37a-d7aa8000d594 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:11.008099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:11.024785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59f8df-9991-4b17-a37a-d7aa8000d594 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:11.059069 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d73fd-9eac-4f02-b075-bffd6a0ad6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:11.059730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d73fd-9eac-4f02-b075-bffd6a0ad6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:11.060000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d73fd-9eac-4f02-b075-bffd6a0ad6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:11.060261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d73fd-9eac-4f02-b075-bffd6a0ad6f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:11.070461 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d73fd-9eac-4f02-b075-bffd6a0ad6f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:11.072075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d73fd-9eac-4f02-b075-bffd6a0ad6f0 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-5e1d73fd-9eac-4f02-b075-bffd6a0ad6f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:58:11.087250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:11.088474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:58:11.089776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:58:11.089776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:11.089776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:58:11.089776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 615/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:11.094693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1650f-49dc-4936-9ad1-e6bcb746a8d2 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:11.202397 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e1650f-49dc-4936-9ad1-e6bcb746a8d2 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:11.212796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:11.238784 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfd4a9-0cc9-46ae-83d4-e057a73c72d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:11.239470 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfd4a9-0cc9-46ae-83d4-e057a73c72d4 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:11.239714 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfd4a9-0cc9-46ae-83d4-e057a73c72d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:11.239981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfd4a9-0cc9-46ae-83d4-e057a73c72d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:11.240619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfd4a9-0cc9-46ae-83d4-e057a73c72d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:11.240916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfd4a9-0cc9-46ae-83d4-e057a73c72d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:11.242308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfd4a9-0cc9-46ae-83d4-e057a73c72d4 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:11.259396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 623/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:11.265179 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d73fd-9eac-4f02-b075-bffd6a0ad6f0 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:11.370583 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d73fd-9eac-4f02-b075-bffd6a0ad6f0 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:11.408484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d8afb-41b6-498a-8356-e383f7cd6959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:11.409147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d8afb-41b6-498a-8356-e383f7cd6959 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:11.409428 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d8afb-41b6-498a-8356-e383f7cd6959 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:11.409703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d8afb-41b6-498a-8356-e383f7cd6959 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:11.413107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:11.419112 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d8afb-41b6-498a-8356-e383f7cd6959 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:11.420800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d8afb-41b6-498a-8356-e383f7cd6959 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-2f2d8afb-41b6-498a-8356-e383f7cd6959 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:58:11.433977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:11.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:11.436444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:11.436444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:11.436444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:58:11.436444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:58:11.436444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:58:11.436444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:58:11.436444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:58:11.436444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:58:11.436444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 15:58:11.436444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:58:11.436444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 616/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:11.441440 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfd4a9-0cc9-46ae-83d4-e057a73c72d4 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:11.548480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfd4a9-0cc9-46ae-83d4-e057a73c72d4 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:11.587985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3d16e-972e-45a2-945b-b7a07b820aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:11.588717 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3d16e-972e-45a2-945b-b7a07b820aaa None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:11.589058 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3d16e-972e-45a2-945b-b7a07b820aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:11.589384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3d16e-972e-45a2-945b-b7a07b820aaa None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:11.590393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3d16e-972e-45a2-945b-b7a07b820aaa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:11.590705 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3d16e-972e-45a2-945b-b7a07b820aaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:11.592671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee3d16e-972e-45a2-945b-b7a07b820aaa None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:11.620399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 624/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Thu Sep 8 15:58:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:11.621722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d8afb-41b6-498a-8356-e383f7cd6959 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:11.735823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d8afb-41b6-498a-8356-e383f7cd6959 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:11.774033 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5d739-d567-42c1-bb1d-a64c472be484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:11.775511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5d739-d567-42c1-bb1d-a64c472be484 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:11.775511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5d739-d567-42c1-bb1d-a64c472be484 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:11.775511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd5d739-d567-42c1-bb1d-a64c472be484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:11.842729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:11.850446 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5d739-d567-42c1-bb1d-a64c472be484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:11.851953 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd5d739-d567-42c1-bb1d-a64c472be484 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:11.965245 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 617/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:58:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:58:11.971655 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3d16e-972e-45a2-945b-b7a07b820aaa None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:12.000807 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:12.078710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3d16e-972e-45a2-945b-b7a07b820aaa None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:12.113982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911d137-f86b-44fd-97f5-2bbf9fed89e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:12.114611 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911d137-f86b-44fd-97f5-2bbf9fed89e6 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:12.114882 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911d137-f86b-44fd-97f5-2bbf9fed89e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:12.115149 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911d137-f86b-44fd-97f5-2bbf9fed89e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:12.125805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911d137-f86b-44fd-97f5-2bbf9fed89e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:12.127397 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911d137-f86b-44fd-97f5-2bbf9fed89e6 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-4911d137-f86b-44fd-97f5-2bbf9fed89e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:58:12.146035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:12.147222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:58:12.148467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:58:12.148467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:12.148467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:12.148467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 625/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:12.153811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5d739-d567-42c1-bb1d-a64c472be484 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:12.260972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:12.267468 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd5d739-d567-42c1-bb1d-a64c472be484 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:12.307691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d5a72-ce73-403e-8563-daabc9cea180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:12.308412 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d5a72-ce73-403e-8563-daabc9cea180 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:12.308831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d5a72-ce73-403e-8563-daabc9cea180 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:12.309151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d5a72-ce73-403e-8563-daabc9cea180 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:12.309843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d5a72-ce73-403e-8563-daabc9cea180 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:12.310174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d5a72-ce73-403e-8563-daabc9cea180 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:12.311973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d5a72-ce73-403e-8563-daabc9cea180 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:12.326163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 618/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:12.332735 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911d137-f86b-44fd-97f5-2bbf9fed89e6 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:12.420156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:12.444877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911d137-f86b-44fd-97f5-2bbf9fed89e6 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:12.482853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc32db-4979-451f-80d4-47635bc5f1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:12.483591 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc32db-4979-451f-80d4-47635bc5f1dd None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:12.483939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc32db-4979-451f-80d4-47635bc5f1dd None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:12.484224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc32db-4979-451f-80d4-47635bc5f1dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:12.491498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc32db-4979-451f-80d4-47635bc5f1dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:12.493197 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc32db-4979-451f-80d4-47635bc5f1dd None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:12.506789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-3bfc32db-4979-451f-80d4-47635bc5f1dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:58:12.506789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:12.506789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:12.506789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:12.506789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:12.506789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:12.506789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:12.506789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:12.506789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:12.506789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:12.506789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:12.506789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:12.506789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:58:12.506789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:58:12.506789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:12.508095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:12.509103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:58:12.509103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:58:12.509103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:58:12.509103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:58:12.509103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:58:12.509103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:58:12.509103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:58:12.509103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:12.509103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 626/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:58:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:12.514497 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d5a72-ce73-403e-8563-daabc9cea180 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:12.628149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d5a72-ce73-403e-8563-daabc9cea180 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:12.667242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c46d70-6f21-4c7b-93bc-c266597a4d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:12.667950 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c46d70-6f21-4c7b-93bc-c266597a4d59 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:12.668212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c46d70-6f21-4c7b-93bc-c266597a4d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:12.668485 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c46d70-6f21-4c7b-93bc-c266597a4d59 None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:12.669249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c46d70-6f21-4c7b-93bc-c266597a4d59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:12.669552 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c46d70-6f21-4c7b-93bc-c266597a4d59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:12.671066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c46d70-6f21-4c7b-93bc-c266597a4d59 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:12.683967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 619/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:58:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:12.690424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc32db-4979-451f-80d4-47635bc5f1dd None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:12.804650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc32db-4979-451f-80d4-47635bc5f1dd None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:12.840541 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c50587-0c04-4bad-8910-e5e416e1d4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:12.841432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c50587-0c04-4bad-8910-e5e416e1d4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:12.841705 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c50587-0c04-4bad-8910-e5e416e1d4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:12.841967 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c50587-0c04-4bad-8910-e5e416e1d4b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:12.842689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c50587-0c04-4bad-8910-e5e416e1d4b2 None admin] RBAC: Adding query filter params (role.id=b256252b45e541a491376a134f502183, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=dd28bb31b55e45668212c9ab69ed3ac3) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:12.842999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c50587-0c04-4bad-8910-e5e416e1d4b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:12.844645 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c50587-0c04-4bad-8910-e5e416e1d4b2 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:12.850528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 627/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:58:12 2022] GET /identity/v3/role_assignments?user.id=dd28bb31b55e45668212c9ab69ed3ac3&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=b256252b45e541a491376a134f502183 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:58:15.240108 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3739e-c235-47f2-9717-da7f0ef592f4 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:15.240683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3739e-c235-47f2-9717-da7f0ef592f4 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:15.240683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3739e-c235-47f2-9717-da7f0ef592f4 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:15.242014 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 620/1247] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:58:15.247325 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ddc74-90d1-46f7-bf1a-2e7ba8d5365f None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:15.247781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ddc74-90d1-46f7-bf1a-2e7ba8d5365f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:15.247992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ddc74-90d1-46f7-bf1a-2e7ba8d5365f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:15.254946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:15.269118 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-374ddc74-90d1-46f7-bf1a-2e7ba8d5365f None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:58:15.270283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-374ddc74-90d1-46f7-bf1a-2e7ba8d5365f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:58:15.343921 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 628/1248] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:58:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:58:15.654549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3739e-c235-47f2-9717-da7f0ef592f4 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:15.676719 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:15.767381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3739e-c235-47f2-9717-da7f0ef592f4 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:15.802532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234096c6-19dd-422d-b880-f85064afd504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:15.803141 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234096c6-19dd-422d-b880-f85064afd504 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:15.803419 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234096c6-19dd-422d-b880-f85064afd504 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:15.803706 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234096c6-19dd-422d-b880-f85064afd504 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:15.815103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234096c6-19dd-422d-b880-f85064afd504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-234096c6-19dd-422d-b880-f85064afd504 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:58:15.820981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:15.822187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:15.823369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:15.823369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:15.823369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:15.823369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:15.823369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:58:15.823369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:58:15.823369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:58:15.823369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:58:15.823369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:58:15.823369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:58:15.823369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:58:15.823369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:58:15.823369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 621/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:58:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:15.828823 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ddc74-90d1-46f7-bf1a-2e7ba8d5365f None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:15.943867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ddc74-90d1-46f7-bf1a-2e7ba8d5365f None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:15.974338 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:15.979918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f244bbe-59b8-48db-9f23-8e5edd751efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:15.980537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f244bbe-59b8-48db-9f23-8e5edd751efa None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:15.980813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f244bbe-59b8-48db-9f23-8e5edd751efa None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:15.981084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f244bbe-59b8-48db-9f23-8e5edd751efa None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:15.981759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f244bbe-59b8-48db-9f23-8e5edd751efa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:15.982066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f244bbe-59b8-48db-9f23-8e5edd751efa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:15.983477 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f244bbe-59b8-48db-9f23-8e5edd751efa None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:15.992702 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 629/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:58:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:58:15.998741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234096c6-19dd-422d-b880-f85064afd504 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:16.087727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:16.116744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234096c6-19dd-422d-b880-f85064afd504 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:16.151761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba3840-2e6a-4291-94f2-f0b5fe757775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:16.152492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba3840-2e6a-4291-94f2-f0b5fe757775 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:16.152760 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba3840-2e6a-4291-94f2-f0b5fe757775 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:16.153039 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba3840-2e6a-4291-94f2-f0b5fe757775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:16.164315 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba3840-2e6a-4291-94f2-f0b5fe757775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:16.165769 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ba3840-2e6a-4291-94f2-f0b5fe757775 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-69ba3840-2e6a-4291-94f2-f0b5fe757775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:58:16.182604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:16.188564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:58:16.190260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:58:16.190260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:16.190260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:58:16.190260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 622/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:16.191809 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f244bbe-59b8-48db-9f23-8e5edd751efa None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:16.301602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f244bbe-59b8-48db-9f23-8e5edd751efa None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:16.338284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f35992-5da5-47e5-b0b4-14d3397c380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:16.338873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f35992-5da5-47e5-b0b4-14d3397c380c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:16.339155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f35992-5da5-47e5-b0b4-14d3397c380c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:16.339437 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f35992-5da5-47e5-b0b4-14d3397c380c None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:16.340269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f35992-5da5-47e5-b0b4-14d3397c380c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:16.340594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f35992-5da5-47e5-b0b4-14d3397c380c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:16.342093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f35992-5da5-47e5-b0b4-14d3397c380c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:16.355282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 630/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:16.361510 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba3840-2e6a-4291-94f2-f0b5fe757775 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:16.476562 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ba3840-2e6a-4291-94f2-f0b5fe757775 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:16.505950 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:16.514678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55b5b9-9432-4059-bed0-2fc11fc4af73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:16.515312 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55b5b9-9432-4059-bed0-2fc11fc4af73 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:16.515723 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55b5b9-9432-4059-bed0-2fc11fc4af73 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:16.515994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55b5b9-9432-4059-bed0-2fc11fc4af73 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:16.523885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55b5b9-9432-4059-bed0-2fc11fc4af73 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:16.525341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55b5b9-9432-4059-bed0-2fc11fc4af73 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-0a55b5b9-9432-4059-bed0-2fc11fc4af73 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:58:16.537751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:16.538894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:16.540103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:16.540103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:16.540103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:58:16.540103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:58:16.540103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:58:16.540103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:58:16.540103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:58:16.540103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:58:16.540103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 15:58:16.540103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:58:16.540103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 623/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:16.545370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f35992-5da5-47e5-b0b4-14d3397c380c None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:16.609880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:16.661170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f35992-5da5-47e5-b0b4-14d3397c380c None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:16.698325 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f32cd-09b1-41a4-a4fd-ac8f5d0c05eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:16.698948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f32cd-09b1-41a4-a4fd-ac8f5d0c05eb None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:16.699214 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f32cd-09b1-41a4-a4fd-ac8f5d0c05eb None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:16.699513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f32cd-09b1-41a4-a4fd-ac8f5d0c05eb None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:16.700256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f32cd-09b1-41a4-a4fd-ac8f5d0c05eb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:16.700549 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f32cd-09b1-41a4-a4fd-ac8f5d0c05eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:16.701989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f32cd-09b1-41a4-a4fd-ac8f5d0c05eb None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:16.721363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 631/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Thu Sep 8 15:58:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:16.728099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55b5b9-9432-4059-bed0-2fc11fc4af73 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:16.839251 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55b5b9-9432-4059-bed0-2fc11fc4af73 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:16.874976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae69b98-006c-4584-aa88-88ecd21066f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:16.875509 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae69b98-006c-4584-aa88-88ecd21066f3 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:16.875802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae69b98-006c-4584-aa88-88ecd21066f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:16.876065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae69b98-006c-4584-aa88-88ecd21066f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:16.935549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:16.950117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae69b98-006c-4584-aa88-88ecd21066f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:16.951621 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae69b98-006c-4584-aa88-88ecd21066f3 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:17.057595 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 624/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:58:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:58:17.064039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f32cd-09b1-41a4-a4fd-ac8f5d0c05eb None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:17.174570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f32cd-09b1-41a4-a4fd-ac8f5d0c05eb None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:17.190402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:17.213673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27a495-009f-4b82-91dd-637441e19c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:17.214315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27a495-009f-4b82-91dd-637441e19c3c None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:17.214605 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27a495-009f-4b82-91dd-637441e19c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:17.214868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27a495-009f-4b82-91dd-637441e19c3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:17.224898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27a495-009f-4b82-91dd-637441e19c3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:17.226332 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27a495-009f-4b82-91dd-637441e19c3c None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-6b27a495-009f-4b82-91dd-637441e19c3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:58:17.241856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:17.243032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:58:17.244276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:58:17.244276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:17.244276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:17.244276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 632/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:17.250537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae69b98-006c-4584-aa88-88ecd21066f3 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:17.338325 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:17.352641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae69b98-006c-4584-aa88-88ecd21066f3 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:17.386486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db045a63-9e30-4235-bbd4-2d017c4ea2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:17.387099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db045a63-9e30-4235-bbd4-2d017c4ea2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:17.387355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db045a63-9e30-4235-bbd4-2d017c4ea2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:17.387703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db045a63-9e30-4235-bbd4-2d017c4ea2c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:17.388342 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db045a63-9e30-4235-bbd4-2d017c4ea2c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:17.388637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db045a63-9e30-4235-bbd4-2d017c4ea2c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:17.390007 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db045a63-9e30-4235-bbd4-2d017c4ea2c0 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:17.401416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 625/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:17.407671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27a495-009f-4b82-91dd-637441e19c3c None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:17.510535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27a495-009f-4b82-91dd-637441e19c3c None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:17.545534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37876e2-3650-4a8c-a1f9-1956e278d463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:17.546150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37876e2-3650-4a8c-a1f9-1956e278d463 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:17.546420 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37876e2-3650-4a8c-a1f9-1956e278d463 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:17.546705 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37876e2-3650-4a8c-a1f9-1956e278d463 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:17.553897 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37876e2-3650-4a8c-a1f9-1956e278d463 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:17.555335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37876e2-3650-4a8c-a1f9-1956e278d463 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:17.566760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-b37876e2-3650-4a8c-a1f9-1956e278d463 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:58:17.566760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:17.566760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:17.566760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:17.566760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:17.566760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:17.566760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:17.566760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:17.566760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:17.566760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:17.566760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:17.566760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:17.566760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:58:17.566760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:58:17.566760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:17.567838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:17.568938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:58:17.568938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:58:17.568938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:58:17.568938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:58:17.568938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:58:17.568938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:58:17.568938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:58:17.568938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:17.568938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 633/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:58:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:17.575061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db045a63-9e30-4235-bbd4-2d017c4ea2c0 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:17.697279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db045a63-9e30-4235-bbd4-2d017c4ea2c0 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:17.732401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99c763-da17-42b0-ad33-60b3dfa84ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:17.732992 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99c763-da17-42b0-ad33-60b3dfa84ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:17.733262 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99c763-da17-42b0-ad33-60b3dfa84ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:17.733528 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99c763-da17-42b0-ad33-60b3dfa84ce7 None admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:17.734233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99c763-da17-42b0-ad33-60b3dfa84ce7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:17.734554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99c763-da17-42b0-ad33-60b3dfa84ce7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:17.736010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99c763-da17-42b0-ad33-60b3dfa84ce7 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:17.747900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 626/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:58:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:17.754429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37876e2-3650-4a8c-a1f9-1956e278d463 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:17.782045 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:17.858221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37876e2-3650-4a8c-a1f9-1956e278d463 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:17.892400 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf050f7-0e08-47d3-aded-9a3a48a358ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:17.892983 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf050f7-0e08-47d3-aded-9a3a48a358ce None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:17.893256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf050f7-0e08-47d3-aded-9a3a48a358ce None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:17.893533 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf050f7-0e08-47d3-aded-9a3a48a358ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:17.894231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf050f7-0e08-47d3-aded-9a3a48a358ce None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=dd28bb31b55e45668212c9ab69ed3ac3) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:17.894534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf050f7-0e08-47d3-aded-9a3a48a358ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:17.895990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf050f7-0e08-47d3-aded-9a3a48a358ce None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:17.901450 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 634/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:58:17 2022] GET /identity/v3/role_assignments?user.id=dd28bb31b55e45668212c9ab69ed3ac3&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 15:58:20.018171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cff20f-9c8d-491e-bc33-a0da5b46ff92 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:20.019081 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cff20f-9c8d-491e-bc33-a0da5b46ff92 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:20.019081 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cff20f-9c8d-491e-bc33-a0da5b46ff92 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:20.021132 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 627/1261] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:58:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:58:20.030110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce046dd6-efeb-449f-84ac-803dc2e70a4b None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:20.030845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce046dd6-efeb-449f-84ac-803dc2e70a4b None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:20.030966 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce046dd6-efeb-449f-84ac-803dc2e70a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:20.055273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-ce046dd6-efeb-449f-84ac-803dc2e70a4b None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:58:20.056574 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ce046dd6-efeb-449f-84ac-803dc2e70a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:58:20.127207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 635/1262] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:58:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:58:20.401271 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cff20f-9c8d-491e-bc33-a0da5b46ff92 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:20.414329 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:20.516820 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cff20f-9c8d-491e-bc33-a0da5b46ff92 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:20.551759 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3420c8-3846-4414-b976-faaf985d2b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:20.552377 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3420c8-3846-4414-b976-faaf985d2b2d None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:20.552649 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3420c8-3846-4414-b976-faaf985d2b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:20.552908 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3420c8-3846-4414-b976-faaf985d2b2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:20.563569 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3420c8-3846-4414-b976-faaf985d2b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-7d3420c8-3846-4414-b976-faaf985d2b2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:58:20.568311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:20.569374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:20.570547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:20.570547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:20.570547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:20.570547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:20.570547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:58:20.570547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:58:20.570547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:58:20.570547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:58:20.570547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:58:20.570547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:58:20.570547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:58:20.570547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:58:20.570547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 628/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:58:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:20.575762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce046dd6-efeb-449f-84ac-803dc2e70a4b None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:20.598959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:20.688777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce046dd6-efeb-449f-84ac-803dc2e70a4b None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:20.736647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa2271-5ba6-45d2-b223-9f0551d8bcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:20.737250 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa2271-5ba6-45d2-b223-9f0551d8bcbc None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:20.737510 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa2271-5ba6-45d2-b223-9f0551d8bcbc None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:20.737773 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa2271-5ba6-45d2-b223-9f0551d8bcbc None admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:20.738436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa2271-5ba6-45d2-b223-9f0551d8bcbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:20.738730 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa2271-5ba6-45d2-b223-9f0551d8bcbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:20.740135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa2271-5ba6-45d2-b223-9f0551d8bcbc None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:20.748659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 636/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:58:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:20.755300 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3420c8-3846-4414-b976-faaf985d2b2d None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:20.832886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:20.878199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3420c8-3846-4414-b976-faaf985d2b2d None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:20.914716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcfe29-0180-41c4-abf2-5891339bccab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:20.915343 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcfe29-0180-41c4-abf2-5891339bccab None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:20.915648 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcfe29-0180-41c4-abf2-5891339bccab None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:20.915917 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcfe29-0180-41c4-abf2-5891339bccab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:20.926288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabcfe29-0180-41c4-abf2-5891339bccab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:20.927781 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabcfe29-0180-41c4-abf2-5891339bccab None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-aabcfe29-0180-41c4-abf2-5891339bccab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:58:20.942230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:20.943339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:58:20.944285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:58:20.944285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:20.944285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:58:20.944285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 629/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:20.949265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa2271-5ba6-45d2-b223-9f0551d8bcbc None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:21.031643 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:21.058154 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa2271-5ba6-45d2-b223-9f0551d8bcbc None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:21.094920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066ef1d-e84e-4a9d-a860-413879c2add4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:21.095509 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066ef1d-e84e-4a9d-a860-413879c2add4 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:21.095810 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066ef1d-e84e-4a9d-a860-413879c2add4 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:21.096074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066ef1d-e84e-4a9d-a860-413879c2add4 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:21.096709 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066ef1d-e84e-4a9d-a860-413879c2add4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:21.097005 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066ef1d-e84e-4a9d-a860-413879c2add4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:21.098371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066ef1d-e84e-4a9d-a860-413879c2add4 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:21.110984 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 637/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:21.117519 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcfe29-0180-41c4-abf2-5891339bccab None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:21.221619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabcfe29-0180-41c4-abf2-5891339bccab None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:21.241597 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:21.258025 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277d3e9-6d19-410d-a5c8-6fd8e3cd3d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:21.258656 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277d3e9-6d19-410d-a5c8-6fd8e3cd3d81 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:21.258927 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277d3e9-6d19-410d-a5c8-6fd8e3cd3d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:21.259191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7277d3e9-6d19-410d-a5c8-6fd8e3cd3d81 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:21.266783 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277d3e9-6d19-410d-a5c8-6fd8e3cd3d81 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:21.268253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7277d3e9-6d19-410d-a5c8-6fd8e3cd3d81 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-7277d3e9-6d19-410d-a5c8-6fd8e3cd3d81 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:58:21.279486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:21.280558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:21.281956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:21.281956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:21.281956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:58:21.281956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:58:21.281956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:58:21.281956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:58:21.281956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:58:21.281956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:58:21.281956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 15:58:21.281956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:58:21.281956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 630/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:21.287241 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066ef1d-e84e-4a9d-a860-413879c2add4 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:21.396418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066ef1d-e84e-4a9d-a860-413879c2add4 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:21.431810 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ab246-4dc8-4fd5-a4c7-35bb62fcf831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:21.432410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ab246-4dc8-4fd5-a4c7-35bb62fcf831 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:21.432665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ab246-4dc8-4fd5-a4c7-35bb62fcf831 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:21.432943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ab246-4dc8-4fd5-a4c7-35bb62fcf831 None admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:21.433632 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ab246-4dc8-4fd5-a4c7-35bb62fcf831 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:21.433940 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ab246-4dc8-4fd5-a4c7-35bb62fcf831 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:21.435382 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ab246-4dc8-4fd5-a4c7-35bb62fcf831 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:21.444590 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:21.454209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 638/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Thu Sep 8 15:58:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:21.462934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277d3e9-6d19-410d-a5c8-6fd8e3cd3d81 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:21.579525 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7277d3e9-6d19-410d-a5c8-6fd8e3cd3d81 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:21.617964 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d665c4-a720-4167-84eb-72f0eee84dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:21.618602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d665c4-a720-4167-84eb-72f0eee84dee None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:21.618880 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d665c4-a720-4167-84eb-72f0eee84dee None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:21.619150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d665c4-a720-4167-84eb-72f0eee84dee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:21.630749 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d665c4-a720-4167-84eb-72f0eee84dee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:21.632221 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d665c4-a720-4167-84eb-72f0eee84dee None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-13d665c4-a720-4167-84eb-72f0eee84dee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:58:21.647555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:21.648644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:58:21.649783 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:58:21.649783 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:21.649783 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:58:21.651642 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 631/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:21.654669 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ab246-4dc8-4fd5-a4c7-35bb62fcf831 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:21.757461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ab246-4dc8-4fd5-a4c7-35bb62fcf831 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:21.802481 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241a93f-dd3d-41b0-b904-1a1773870c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:21.803101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241a93f-dd3d-41b0-b904-1a1773870c62 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:21.803376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241a93f-dd3d-41b0-b904-1a1773870c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:21.803676 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241a93f-dd3d-41b0-b904-1a1773870c62 None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:21.804306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241a93f-dd3d-41b0-b904-1a1773870c62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:21.804605 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241a93f-dd3d-41b0-b904-1a1773870c62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:21.806018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241a93f-dd3d-41b0-b904-1a1773870c62 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:21.819316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 639/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:21.827165 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d665c4-a720-4167-84eb-72f0eee84dee None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:21.861542 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:21.931749 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d665c4-a720-4167-84eb-72f0eee84dee None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:21.966502 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651b9e3-04a5-464b-8288-5cebc9d029d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:21.967117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651b9e3-04a5-464b-8288-5cebc9d029d3 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:21.967392 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651b9e3-04a5-464b-8288-5cebc9d029d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:21.967692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651b9e3-04a5-464b-8288-5cebc9d029d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:21.974663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651b9e3-04a5-464b-8288-5cebc9d029d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:21.976152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651b9e3-04a5-464b-8288-5cebc9d029d3 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:21.987655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-9651b9e3-04a5-464b-8288-5cebc9d029d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:58:21.987655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:21.987655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:21.987655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:21.987655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:21.987655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:21.987655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:21.987655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:21.987655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:21.987655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:21.987655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:21.987655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:21.987655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:58:21.987655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:58:21.987655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:21.988652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:21.989637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:58:21.989637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:58:21.989637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:58:21.989637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:58:21.989637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:58:21.989637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:58:21.989637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:58:21.989637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:58:21.989637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 632/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:58:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:21.995816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241a93f-dd3d-41b0-b904-1a1773870c62 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:22.069068 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:22.122855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241a93f-dd3d-41b0-b904-1a1773870c62 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:22.158428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2cd88-4edb-4887-9ec8-34d249c774bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:22.159025 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2cd88-4edb-4887-9ec8-34d249c774bd None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:22.159298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2cd88-4edb-4887-9ec8-34d249c774bd None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:22.159603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2cd88-4edb-4887-9ec8-34d249c774bd None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:22.160291 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2cd88-4edb-4887-9ec8-34d249c774bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:22.160594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2cd88-4edb-4887-9ec8-34d249c774bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:22.162034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2cd88-4edb-4887-9ec8-34d249c774bd None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:22.175011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 640/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:58:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:22.182968 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651b9e3-04a5-464b-8288-5cebc9d029d3 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:22.291765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:22.301882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651b9e3-04a5-464b-8288-5cebc9d029d3 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:22.338696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa59d2-71ef-4557-84bc-9e41b88ba9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:22.339348 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa59d2-71ef-4557-84bc-9e41b88ba9c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:22.339660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa59d2-71ef-4557-84bc-9e41b88ba9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:22.339934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa59d2-71ef-4557-84bc-9e41b88ba9c6 None admin] PATH_INFO: `/v3/projects/a825a0b1238d4382995da440944f453b/users/dd28bb31b55e45668212c9ab69ed3ac3/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:22.341870 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa59d2-71ef-4557-84bc-9e41b88ba9c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a825a0b1238d4382995da440944f453b, user_id=dd28bb31b55e45668212c9ab69ed3ac3, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:22.343311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa59d2-71ef-4557-84bc-9e41b88ba9c6 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:22.359400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 633/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Thu Sep 8 15:58:22 2022] PUT /identity/v3/projects/a825a0b1238d4382995da440944f453b/users/dd28bb31b55e45668212c9ab69ed3ac3/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 15:58:24.478342 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30e45d-b5ad-4d3d-89dc-d859a888168a None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:24.479188 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30e45d-b5ad-4d3d-89dc-d859a888168a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:24.479188 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c30e45d-b5ad-4d3d-89dc-d859a888168a None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:24.481258 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 641/1274] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:58:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:58:24.487269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3178f2-66c7-410f-844a-45fb42a9fc4a None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:24.487819 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3178f2-66c7-410f-844a-45fb42a9fc4a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:24.487819 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3178f2-66c7-410f-844a-45fb42a9fc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:24.507793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-eb3178f2-66c7-410f-844a-45fb42a9fc4a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:58:24.508994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-eb3178f2-66c7-410f-844a-45fb42a9fc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:58:24.583330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 634/1275] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:58:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:58:24.846980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30e45d-b5ad-4d3d-89dc-d859a888168a None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:24.955371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c30e45d-b5ad-4d3d-89dc-d859a888168a None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:24.973208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:24.992594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa2d5d-93fe-44c4-9526-4359de3f9a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:24.993201 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa2d5d-93fe-44c4-9526-4359de3f9a17 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:24.993474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa2d5d-93fe-44c4-9526-4359de3f9a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:24.993739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa2d5d-93fe-44c4-9526-4359de3f9a17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:25.004918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa2d5d-93fe-44c4-9526-4359de3f9a17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-e9aa2d5d-93fe-44c4-9526-4359de3f9a17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 15:58:25.009691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:25.010727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:25.011685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:25.011685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:25.011685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:25.011685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:25.011685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 15:58:25.011685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 15:58:25.011685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 15:58:25.011685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 15:58:25.011685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 15:58:25.011685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 15:58:25.011685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 15:58:25.011685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:25.011685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 642/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Thu Sep 8 15:58:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:25.017089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3178f2-66c7-410f-844a-45fb42a9fc4a None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:25.099435 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:25.121190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3178f2-66c7-410f-844a-45fb42a9fc4a None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:25.154712 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d5324-7c7e-427d-9619-20b86c4a0d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:25.155305 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d5324-7c7e-427d-9619-20b86c4a0d86 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:25.155608 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d5324-7c7e-427d-9619-20b86c4a0d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:25.155925 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d5324-7c7e-427d-9619-20b86c4a0d86 None admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:25.156580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d5324-7c7e-427d-9619-20b86c4a0d86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:25.156874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d5324-7c7e-427d-9619-20b86c4a0d86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:25.158232 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d5324-7c7e-427d-9619-20b86c4a0d86 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:25.166212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 635/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Thu Sep 8 15:58:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:25.172743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa2d5d-93fe-44c4-9526-4359de3f9a17 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:25.285443 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa2d5d-93fe-44c4-9526-4359de3f9a17 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:25.321819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee686c-ed9a-4839-a132-7aea83feda08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:25.322431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee686c-ed9a-4839-a132-7aea83feda08 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:25.322710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee686c-ed9a-4839-a132-7aea83feda08 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:25.322971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee686c-ed9a-4839-a132-7aea83feda08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:25.333826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee686c-ed9a-4839-a132-7aea83feda08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:25.335277 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee686c-ed9a-4839-a132-7aea83feda08 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-ebee686c-ed9a-4839-a132-7aea83feda08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:58:25.350127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:25.351191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:58:25.352222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:58:25.352222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:25.352222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:25.352222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 643/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:25.357281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d5324-7c7e-427d-9619-20b86c4a0d86 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:25.461521 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d5324-7c7e-427d-9619-20b86c4a0d86 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:25.495740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fba1bd-f48c-4862-93ff-6490d4bf6bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:25.496337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fba1bd-f48c-4862-93ff-6490d4bf6bfb None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:25.496589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fba1bd-f48c-4862-93ff-6490d4bf6bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:25.496848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fba1bd-f48c-4862-93ff-6490d4bf6bfb None admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:25.497483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fba1bd-f48c-4862-93ff-6490d4bf6bfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:25.497775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fba1bd-f48c-4862-93ff-6490d4bf6bfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:25.499134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fba1bd-f48c-4862-93ff-6490d4bf6bfb None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:25.510825 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 636/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:25.517521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee686c-ed9a-4839-a132-7aea83feda08 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:25.560077 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:25.635472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee686c-ed9a-4839-a132-7aea83feda08 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:25.670935 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8f5e7-867d-48bd-a007-139f6a6cb378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:25.671607 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8f5e7-867d-48bd-a007-139f6a6cb378 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:25.671886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8f5e7-867d-48bd-a007-139f6a6cb378 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:25.672156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8f5e7-867d-48bd-a007-139f6a6cb378 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:25.679887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8f5e7-867d-48bd-a007-139f6a6cb378 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:25.681377 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8f5e7-867d-48bd-a007-139f6a6cb378 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-6fb8f5e7-867d-48bd-a007-139f6a6cb378 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 15:58:25.692900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:25.695573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:25.696798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:25.696798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:25.696798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 15:58:25.696798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 15:58:25.696798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 15:58:25.696798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 15:58:25.696798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 15:58:25.696798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 15:58:25.696798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 15:58:25.696798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:25.696798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 644/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:25.700153 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fba1bd-f48c-4862-93ff-6490d4bf6bfb None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:25.704225 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:25.804005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fba1bd-f48c-4862-93ff-6490d4bf6bfb None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:25.839541 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab47ef-b9b2-4b80-ae70-532a8b090c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:25.840137 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab47ef-b9b2-4b80-ae70-532a8b090c95 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:25.840411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab47ef-b9b2-4b80-ae70-532a8b090c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:25.840679 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab47ef-b9b2-4b80-ae70-532a8b090c95 None admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:25.841367 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab47ef-b9b2-4b80-ae70-532a8b090c95 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:25.841714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab47ef-b9b2-4b80-ae70-532a8b090c95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:25.843141 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dab47ef-b9b2-4b80-ae70-532a8b090c95 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:25.860144 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 637/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Thu Sep 8 15:58:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:25.869421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8f5e7-867d-48bd-a007-139f6a6cb378 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:25.976455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:25.987674 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8f5e7-867d-48bd-a007-139f6a6cb378 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:26.024106 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde49fb8-b36c-4435-b4c7-314e2d239334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:26.024697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde49fb8-b36c-4435-b4c7-314e2d239334 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:26.024964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde49fb8-b36c-4435-b4c7-314e2d239334 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:26.025227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde49fb8-b36c-4435-b4c7-314e2d239334 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:26.108723 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde49fb8-b36c-4435-b4c7-314e2d239334 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:26.110267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde49fb8-b36c-4435-b4c7-314e2d239334 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:26.202637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:26.214520 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 645/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Thu Sep 8 15:58:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 15:58:26.220847 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab47ef-b9b2-4b80-ae70-532a8b090c95 None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:26.286696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:26.326985 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dab47ef-b9b2-4b80-ae70-532a8b090c95 None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:26.362776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75669e3c-fde4-40fd-9ceb-58c0d5a5c20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:26.363397 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75669e3c-fde4-40fd-9ceb-58c0d5a5c20d None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:26.363707 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75669e3c-fde4-40fd-9ceb-58c0d5a5c20d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:26.363977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75669e3c-fde4-40fd-9ceb-58c0d5a5c20d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:26.374892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75669e3c-fde4-40fd-9ceb-58c0d5a5c20d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:26.376371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75669e3c-fde4-40fd-9ceb-58c0d5a5c20d None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-75669e3c-fde4-40fd-9ceb-58c0d5a5c20d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 15:58:26.391240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:26.392391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 15:58:26.393369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 15:58:26.393369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 15:58:26.393369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:58:26.393369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 638/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 15:58:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:26.398832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde49fb8-b36c-4435-b4c7-314e2d239334 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:26.501301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde49fb8-b36c-4435-b4c7-314e2d239334 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:26.539049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9049d-8a42-4be4-b8ac-610190bae6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:26.539649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9049d-8a42-4be4-b8ac-610190bae6de None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:26.539873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9049d-8a42-4be4-b8ac-610190bae6de None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:26.540164 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9049d-8a42-4be4-b8ac-610190bae6de None admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:26.540782 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9049d-8a42-4be4-b8ac-610190bae6de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:26.541076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9049d-8a42-4be4-b8ac-610190bae6de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:26.542443 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9049d-8a42-4be4-b8ac-610190bae6de None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:26.553697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 646/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Thu Sep 8 15:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:26.562615 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75669e3c-fde4-40fd-9ceb-58c0d5a5c20d None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:26.669545 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75669e3c-fde4-40fd-9ceb-58c0d5a5c20d None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:26.683151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:26.705580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48a7cf-8d80-4f2a-9acd-23f04c46436a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:26.706199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48a7cf-8d80-4f2a-9acd-23f04c46436a None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:26.706798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48a7cf-8d80-4f2a-9acd-23f04c46436a None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:26.707093 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48a7cf-8d80-4f2a-9acd-23f04c46436a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:26.714319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48a7cf-8d80-4f2a-9acd-23f04c46436a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:26.715785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e48a7cf-8d80-4f2a-9acd-23f04c46436a None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:26.727326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application [None req-0e48a7cf-8d80-4f2a-9acd-23f04c46436a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:58:26.727326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:26.727326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:26.727326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:26.727326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:26.727326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:26.727326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:26.727326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:26.727326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:26.727326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:26.727326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:26.727326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:26.727326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 15:58:26.727326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 15:58:26.727326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:26.733128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:26.734370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 15:58:26.734370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 15:58:26.734370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 15:58:26.734370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 15:58:26.734370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 15:58:26.734370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 15:58:26.734370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 15:58:26.734370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: ERROR keystone.server.flask.application Sep 08 15:58:26.734370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 639/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Thu Sep 8 15:58:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:26.734984 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9049d-8a42-4be4-b8ac-610190bae6de None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:26.788781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:26.844348 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9049d-8a42-4be4-b8ac-610190bae6de None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:26.880261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccadbad-b45b-401b-859c-89bf29550b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:26.880856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccadbad-b45b-401b-859c-89bf29550b48 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:26.881128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccadbad-b45b-401b-859c-89bf29550b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:26.881398 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccadbad-b45b-401b-859c-89bf29550b48 None admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:26.882093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccadbad-b45b-401b-859c-89bf29550b48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:26.882393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccadbad-b45b-401b-859c-89bf29550b48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:26.883868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccadbad-b45b-401b-859c-89bf29550b48 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:26.895904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 647/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Thu Sep 8 15:58:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:26.902790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48a7cf-8d80-4f2a-9acd-23f04c46436a None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:27.024980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48a7cf-8d80-4f2a-9acd-23f04c46436a None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:27.069281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fbfca-867d-49e5-9141-750383d85f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:27.069873 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fbfca-867d-49e5-9141-750383d85f3d None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:27.070143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fbfca-867d-49e5-9141-750383d85f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:27.070413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fbfca-867d-49e5-9141-750383d85f3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:27.071117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fbfca-867d-49e5-9141-750383d85f3d None admin] RBAC: Adding query filter params (role.id=bb69e9f6c5ed4649a2810514178a6216, scope.project.id=a825a0b1238d4382995da440944f453b, user.id=dd28bb31b55e45668212c9ab69ed3ac3) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:27.071418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fbfca-867d-49e5-9141-750383d85f3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:27.072858 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fbfca-867d-49e5-9141-750383d85f3d None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:27.078880 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 640/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Thu Sep 8 15:58:26 2022] GET /identity/v3/role_assignments?user.id=dd28bb31b55e45668212c9ab69ed3ac3&scope.project.id=a825a0b1238d4382995da440944f453b&role.id=bb69e9f6c5ed4649a2810514178a6216 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:29.261421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ede33-2824-405b-b917-1b905839deaf None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:29.262028 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ede33-2824-405b-b917-1b905839deaf None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:29.262028 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ede33-2824-405b-b917-1b905839deaf None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:29.263335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 648/1288] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:58:29.269778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff257aec-f63f-413c-b53b-88ba2f2edb2c None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:29.270431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff257aec-f63f-413c-b53b-88ba2f2edb2c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:29.270579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff257aec-f63f-413c-b53b-88ba2f2edb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:29.298892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-ff257aec-f63f-413c-b53b-88ba2f2edb2c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:58:29.300105 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ff257aec-f63f-413c-b53b-88ba2f2edb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:58:29.331742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:29.378839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 641/1289] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:58:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:58:29.649878 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ede33-2824-405b-b917-1b905839deaf None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:29.766979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ede33-2824-405b-b917-1b905839deaf None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:29.777559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:29.810629 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a61fd-58f9-493a-a220-84a50124907e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:29.811250 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a61fd-58f9-493a-a220-84a50124907e None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:29.811564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a61fd-58f9-493a-a220-84a50124907e None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:29.811834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a61fd-58f9-493a-a220-84a50124907e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:29.812420 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a61fd-58f9-493a-a220-84a50124907e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:29.813805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a61fd-58f9-493a-a220-84a50124907e None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-5d3a61fd-58f9-493a-a220-84a50124907e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:29.822870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:29.824218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:29.825160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:29.825160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:29.825160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:29.825160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:29.825160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:29.825160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:29.825160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:29.825160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:29.825160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:29.825160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:29.825160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:58:29.825160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:58:29.825160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 15:58:29.825160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:29.825160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 649/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Sep 8 15:58:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:29.830434 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff257aec-f63f-413c-b53b-88ba2f2edb2c None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:29.933318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff257aec-f63f-413c-b53b-88ba2f2edb2c None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:29.968464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42e5c59-1690-4693-9556-8ebd31aa32b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:29.969069 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e5c59-1690-4693-9556-8ebd31aa32b8 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:29.969342 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e5c59-1690-4693-9556-8ebd31aa32b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:29.969607 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42e5c59-1690-4693-9556-8ebd31aa32b8 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:29.970233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e5c59-1690-4693-9556-8ebd31aa32b8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:29.970530 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e5c59-1690-4693-9556-8ebd31aa32b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:29.971934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42e5c59-1690-4693-9556-8ebd31aa32b8 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:29.977017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 642/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Thu Sep 8 15:58:29 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:29.985386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a61fd-58f9-493a-a220-84a50124907e None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:30.096390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a61fd-58f9-493a-a220-84a50124907e None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:30.131350 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8eb2cf-8c06-4cb9-acdc-6e6aefddee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:30.131987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8eb2cf-8c06-4cb9-acdc-6e6aefddee1a None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:30.132264 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8eb2cf-8c06-4cb9-acdc-6e6aefddee1a None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:30.132537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8eb2cf-8c06-4cb9-acdc-6e6aefddee1a None admin] PATH_INFO: `/v3/services` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:30.133156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8eb2cf-8c06-4cb9-acdc-6e6aefddee1a None admin] RBAC: Adding query filter params (type=placement) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:30.133454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8eb2cf-8c06-4cb9-acdc-6e6aefddee1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:30.134819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8eb2cf-8c06-4cb9-acdc-6e6aefddee1a None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:30.139416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 650/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Thu Sep 8 15:58:29 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:32.212562 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d64d20-327f-4c50-bebd-1bb6ac1695a7 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:32.213367 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d64d20-327f-4c50-bebd-1bb6ac1695a7 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:32.213555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d64d20-327f-4c50-bebd-1bb6ac1695a7 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:32.215709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 643/1293] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:58:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:58:32.221913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d98fb-b29f-4a5e-980e-705e88818a9e None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:32.222344 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d98fb-b29f-4a5e-980e-705e88818a9e None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:32.222445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d98fb-b29f-4a5e-980e-705e88818a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:32.247229 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f55d98fb-b29f-4a5e-980e-705e88818a9e None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:58:32.248881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f55d98fb-b29f-4a5e-980e-705e88818a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:58:32.281673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:32.321979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 651/1294] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:58:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:58:32.586206 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d64d20-327f-4c50-bebd-1bb6ac1695a7 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:32.626556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:32.703401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d64d20-327f-4c50-bebd-1bb6ac1695a7 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:32.739152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d6a6d-73f0-4c6c-887a-c869d2ddce70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:32.739786 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d6a6d-73f0-4c6c-887a-c869d2ddce70 None admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:32.740064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d6a6d-73f0-4c6c-887a-c869d2ddce70 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:32.740334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d6a6d-73f0-4c6c-887a-c869d2ddce70 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:32.741032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d6a6d-73f0-4c6c-887a-c869d2ddce70 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:32.742404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d6a6d-73f0-4c6c-887a-c869d2ddce70 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:32.758976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 644/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Thu Sep 8 15:58:32 2022] POST /identity/v3/services => generated 246 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:58:34.910116 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa70fc-6442-447a-9421-7b76579a7c03 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:34.910734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa70fc-6442-447a-9421-7b76579a7c03 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:34.910734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaa70fc-6442-447a-9421-7b76579a7c03 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:34.912032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 652/1296] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:58:34.918388 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71436bf1-e990-4bd5-8235-8b43e11b49a0 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:34.919119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71436bf1-e990-4bd5-8235-8b43e11b49a0 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:34.919300 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71436bf1-e990-4bd5-8235-8b43e11b49a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:34.945285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-71436bf1-e990-4bd5-8235-8b43e11b49a0 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:58:34.946468 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-71436bf1-e990-4bd5-8235-8b43e11b49a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:58:35.025701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 645/1297] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:58:34 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 15:58:35.302777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa70fc-6442-447a-9421-7b76579a7c03 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:35.420233 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaa70fc-6442-447a-9421-7b76579a7c03 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:35.455894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651978c-5b88-4d6e-b229-60794fc747c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:35.456789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651978c-5b88-4d6e-b229-60794fc747c5 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:35.457061 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651978c-5b88-4d6e-b229-60794fc747c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:35.457328 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d651978c-5b88-4d6e-b229-60794fc747c5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:35.457917 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651978c-5b88-4d6e-b229-60794fc747c5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:35.459302 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d651978c-5b88-4d6e-b229-60794fc747c5 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-d651978c-5b88-4d6e-b229-60794fc747c5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:35.468665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:35.469797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:35.470794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:35.470794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:35.470794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:35.470794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:35.470794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:35.470794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:35.470794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:35.470794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:35.470794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:35.470794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:35.470794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:58:35.470794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:58:35.470794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 15:58:35.470794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:35.470794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 653/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Sep 8 15:58:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:35.475947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71436bf1-e990-4bd5-8235-8b43e11b49a0 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:35.496308 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:35.593062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71436bf1-e990-4bd5-8235-8b43e11b49a0 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:35.636207 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ec1f2-a333-46ee-9dd2-cf48dbe69d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:35.636798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ec1f2-a333-46ee-9dd2-cf48dbe69d1b None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:35.637076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ec1f2-a333-46ee-9dd2-cf48dbe69d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:35.637328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ec1f2-a333-46ee-9dd2-cf48dbe69d1b None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:35.642752 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ec1f2-a333-46ee-9dd2-cf48dbe69d1b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:35.642752 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ec1f2-a333-46ee-9dd2-cf48dbe69d1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:35.642752 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022ec1f2-a333-46ee-9dd2-cf48dbe69d1b None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:35.646146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 646/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Thu Sep 8 15:58:35 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:35.652739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651978c-5b88-4d6e-b229-60794fc747c5 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:35.690724 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:35.761823 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d651978c-5b88-4d6e-b229-60794fc747c5 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:35.798583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5507fe-b6b4-40c1-9f56-21673ef58ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:35.799172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5507fe-b6b4-40c1-9f56-21673ef58ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:35.799473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5507fe-b6b4-40c1-9f56-21673ef58ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:35.799745 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5507fe-b6b4-40c1-9f56-21673ef58ac8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:35.800437 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5507fe-b6b4-40c1-9f56-21673ef58ac8 None admin] RBAC: Adding query filter params (interface=public, service_id=1225c337df96478e9f0470279931be52, region_id=RegionOne) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:35.800735 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5507fe-b6b4-40c1-9f56-21673ef58ac8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:35.802109 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5507fe-b6b4-40c1-9f56-21673ef58ac8 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:35.807260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 654/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Thu Sep 8 15:58:35 2022] GET /identity/v3/endpoints?service_id=1225c337df96478e9f0470279931be52&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:35.813355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ec1f2-a333-46ee-9dd2-cf48dbe69d1b None admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:35.899299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:35.917685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022ec1f2-a333-46ee-9dd2-cf48dbe69d1b None admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:35.963260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3b5e5d-70bb-427a-96f4-bfdb18b32ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:35.964013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b5e5d-70bb-427a-96f4-bfdb18b32ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:35.964178 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b5e5d-70bb-427a-96f4-bfdb18b32ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:35.964449 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3b5e5d-70bb-427a-96f4-bfdb18b32ac0 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:35.965139 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b5e5d-70bb-427a-96f4-bfdb18b32ac0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:35.966517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3b5e5d-70bb-427a-96f4-bfdb18b32ac0 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:35.973726 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 647/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Thu Sep 8 15:58:35 2022] GET /identity/v3/services => generated 2191 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 15:58:38.086019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d3151-ef57-434f-a93a-effce10ffad1 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:38.086940 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d3151-ef57-434f-a93a-effce10ffad1 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:38.086940 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51d3151-ef57-434f-a93a-effce10ffad1 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:38.089090 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 655/1302] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:58:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:58:38.097114 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd28fef-56b8-49a0-896d-81a641ad927d None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:38.097568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd28fef-56b8-49a0-896d-81a641ad927d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:38.097661 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd28fef-56b8-49a0-896d-81a641ad927d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:38.117215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-6bd28fef-56b8-49a0-896d-81a641ad927d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:58:38.118379 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6bd28fef-56b8-49a0-896d-81a641ad927d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:58:38.188910 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 648/1303] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:58:38 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 15:58:38.459974 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d3151-ef57-434f-a93a-effce10ffad1 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:38.566038 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:38.574606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51d3151-ef57-434f-a93a-effce10ffad1 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:38.611916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b20005-8887-4394-b072-d740b50260b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:38.612549 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b20005-8887-4394-b072-d740b50260b0 None admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:38.612826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b20005-8887-4394-b072-d740b50260b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:38.613133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b20005-8887-4394-b072-d740b50260b0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:38.613688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b20005-8887-4394-b072-d740b50260b0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:38.615101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b20005-8887-4394-b072-d740b50260b0 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application [None req-91b20005-8887-4394-b072-d740b50260b0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 15:58:38.624247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 15:58:38.625580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 15:58:38.626648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 15:58:38.626648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application with Lock( Sep 08 15:58:38.626648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 15:58:38.626648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self._enter() Sep 08 15:58:38.626648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 15:58:38.626648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 15:58:38.626648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 15:58:38.626648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application return self.creator() Sep 08 15:58:38.626648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 15:58:38.626648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application created_value = creator( Sep 08 15:58:38.626648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 15:58:38.626648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 15:58:38.626648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 15:58:38.626648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: ERROR keystone.server.flask.application Sep 08 15:58:38.626648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 656/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Thu Sep 8 15:58:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 15:58:38.631527 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd28fef-56b8-49a0-896d-81a641ad927d None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:38.720037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:38.743287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd28fef-56b8-49a0-896d-81a641ad927d None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:38.776684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b31560-0b4e-4411-ac0d-9a2b4ff5b258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:38.777280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b31560-0b4e-4411-ac0d-9a2b4ff5b258 None admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:38.777554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b31560-0b4e-4411-ac0d-9a2b4ff5b258 None admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:38.777804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b31560-0b4e-4411-ac0d-9a2b4ff5b258 None admin] PATH_INFO: `/v3/services` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:38.778420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b31560-0b4e-4411-ac0d-9a2b4ff5b258 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 15:58:38.778718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b31560-0b4e-4411-ac0d-9a2b4ff5b258 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:38.780117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b31560-0b4e-4411-ac0d-9a2b4ff5b258 None admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:38.785187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 649/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Thu Sep 8 15:58:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 15:58:38.791581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b20005-8887-4394-b072-d740b50260b0 None admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:58:38.912289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b20005-8887-4394-b072-d740b50260b0 None admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:58:38.947399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f03a8-fe13-4705-aa1c-21df6c8fc660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:58:38.948020 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f03a8-fe13-4705-aa1c-21df6c8fc660 None admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:58:38.948303 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f03a8-fe13-4705-aa1c-21df6c8fc660 None admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:58:38.948569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f03a8-fe13-4705-aa1c-21df6c8fc660 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:58:38.949236 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f03a8-fe13-4705-aa1c-21df6c8fc660 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:58:38.950607 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f03a8-fe13-4705-aa1c-21df6c8fc660 None admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:58:38.979016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 657/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 15:58:38 2022] POST /identity/v3/endpoints => generated 331 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 15:58:48.782201 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=30 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:58:48.970998 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:59:42.761436 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab9ae8-b8cc-4c46-988f-65cc617fc907 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:42.761886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab9ae8-b8cc-4c46-988f-65cc617fc907 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:42.762049 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab9ae8-b8cc-4c46-988f-65cc617fc907 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:42.763378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 650/1307] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:59:42.769099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18137403-62ac-405a-8aaf-aeec3af3f5e7 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:42.769612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18137403-62ac-405a-8aaf-aeec3af3f5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:42.769612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18137403-62ac-405a-8aaf-aeec3af3f5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:42.886980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-18137403-62ac-405a-8aaf-aeec3af3f5e7 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:59:42.889492 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-18137403-62ac-405a-8aaf-aeec3af3f5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:43.060672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 658/1308] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 15:59:43.351351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dc0fa-e167-43c9-9114-670da5830d71 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:43.351660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dc0fa-e167-43c9-9114-670da5830d71 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:43.351931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742dc0fa-e167-43c9-9114-670da5830d71 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:43.353354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 651/1309] 10.222.0.27 () {54 vars in 965 bytes} [Thu Sep 8 15:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:59:43.360625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81941425-3b10-404c-8358-fe359e5fab42 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:43.361255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81941425-3b10-404c-8358-fe359e5fab42 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:43.361947 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81941425-3b10-404c-8358-fe359e5fab42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:43.466056 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-81941425-3b10-404c-8358-fe359e5fab42 None None] MFA Rules not processed for user `3aae65f9ec394f97aeafad43932dc49c`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:59:43.467619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-81941425-3b10-404c-8358-fe359e5fab42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:43.492942 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:59:43.557098 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 659/1310] 10.222.0.27 () {58 vars in 1062 bytes} [Thu Sep 8 15:59:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 15:59:43.567411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dc0fa-e167-43c9-9114-670da5830d71 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:59:43.569758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-742dc0fa-e167-43c9-9114-670da5830d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:43.696509 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742dc0fa-e167-43c9-9114-670da5830d71 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:59:43.790011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac16874-85c8-47a0-804d-77e8c7b1b464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:59:43.791963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac16874-85c8-47a0-804d-77e8c7b1b464 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:43.792457 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac16874-85c8-47a0-804d-77e8c7b1b464 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:43.792963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac16874-85c8-47a0-804d-77e8c7b1b464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:43.795206 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3ac16874-85c8-47a0-804d-77e8c7b1b464 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:43.859550 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:59:43.937195 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac16874-85c8-47a0-804d-77e8c7b1b464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:59:43.939463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac16874-85c8-47a0-804d-77e8c7b1b464 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:59:43.966338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 652/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 15:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 15:59:46.418281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e60c7-5429-4aca-98ba-9390eabcedb3 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:46.419137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e60c7-5429-4aca-98ba-9390eabcedb3 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:46.419137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e60c7-5429-4aca-98ba-9390eabcedb3 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:46.420943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 660/1312] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:59:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:59:46.427317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d497d-d56b-4ebc-b47b-51639214bf78 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:46.428059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d497d-d56b-4ebc-b47b-51639214bf78 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:46.428154 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1d497d-d56b-4ebc-b47b-51639214bf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:46.528946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-9b1d497d-d56b-4ebc-b47b-51639214bf78 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:59:46.530194 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9b1d497d-d56b-4ebc-b47b-51639214bf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:46.630431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 653/1313] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 15:59:46.913143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e60c7-5429-4aca-98ba-9390eabcedb3 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:59:46.915675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-292e60c7-5429-4aca-98ba-9390eabcedb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:47.038827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e60c7-5429-4aca-98ba-9390eabcedb3 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:59:47.114265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:59:47.116600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9e575-f6e5-462f-97a4-fc565a435225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:59:47.117297 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9e575-f6e5-462f-97a4-fc565a435225 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:47.117582 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9e575-f6e5-462f-97a4-fc565a435225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:47.117852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9e575-f6e5-462f-97a4-fc565a435225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:47.120282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-93f9e575-f6e5-462f-97a4-fc565a435225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:47.237214 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9e575-f6e5-462f-97a4-fc565a435225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:59:47.239533 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9e575-f6e5-462f-97a4-fc565a435225 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:59:47.267775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 661/1314] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 15:59:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 15:59:49.683928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88173d54-c551-4f54-b356-c85e683fc655 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:49.684824 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88173d54-c551-4f54-b356-c85e683fc655 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:49.684824 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88173d54-c551-4f54-b356-c85e683fc655 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:49.686482 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 654/1315] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:59:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:59:49.692018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad5d44-a4fb-4ea2-9822-3f2a0fe38131 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:49.692467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad5d44-a4fb-4ea2-9822-3f2a0fe38131 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:49.692567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ad5d44-a4fb-4ea2-9822-3f2a0fe38131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:49.760392 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-e6ad5d44-a4fb-4ea2-9822-3f2a0fe38131 None None] MFA Rules not processed for user `8ee93dd82ab0424e9e7c9bdcc0d19a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:59:49.761583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e6ad5d44-a4fb-4ea2-9822-3f2a0fe38131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:49.816968 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:59:49.863870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 662/1316] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:59:49 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 15:59:50.143739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88173d54-c551-4f54-b356-c85e683fc655 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:59:50.146227 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-88173d54-c551-4f54-b356-c85e683fc655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:50.162294 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:59:50.276606 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88173d54-c551-4f54-b356-c85e683fc655 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:59:50.355327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629662b-48d5-4731-ad55-ac85f5055faa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:59:50.355966 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629662b-48d5-4731-ad55-ac85f5055faa service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:50.356242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629662b-48d5-4731-ad55-ac85f5055faa service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:50.356515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629662b-48d5-4731-ad55-ac85f5055faa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:50.358172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f629662b-48d5-4731-ad55-ac85f5055faa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:50.425125 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:59:50.474271 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629662b-48d5-4731-ad55-ac85f5055faa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:59:50.475743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629662b-48d5-4731-ad55-ac85f5055faa service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:59:50.493923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 655/1317] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 15:59:50 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 15:59:53.145026 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2a3df-d448-4635-a429-9dbd479aff6b None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:53.145604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2a3df-d448-4635-a429-9dbd479aff6b None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:53.145604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2a3df-d448-4635-a429-9dbd479aff6b None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:53.146901 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 663/1318] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:59:53.153302 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6443c-7edf-467d-ba37-d48ff495b495 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:53.154017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6443c-7edf-467d-ba37-d48ff495b495 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:53.154167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6443c-7edf-467d-ba37-d48ff495b495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:53.232700 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-8bc6443c-7edf-467d-ba37-d48ff495b495 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:59:53.233946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-8bc6443c-7edf-467d-ba37-d48ff495b495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:53.350276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 656/1319] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:59:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 15:59:53.634141 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2a3df-d448-4635-a429-9dbd479aff6b None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:59:53.635721 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-bff2a3df-d448-4635-a429-9dbd479aff6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:53.747808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff2a3df-d448-4635-a429-9dbd479aff6b None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:59:53.827364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f422485-477f-4684-9996-199e1904770c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:59:53.828016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f422485-477f-4684-9996-199e1904770c service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:53.828612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f422485-477f-4684-9996-199e1904770c service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:53.828912 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f422485-477f-4684-9996-199e1904770c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:53.830540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8f422485-477f-4684-9996-199e1904770c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:53.837692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:59:53.960712 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f422485-477f-4684-9996-199e1904770c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:59:53.962055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f422485-477f-4684-9996-199e1904770c service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:59:53.980522 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 664/1320] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 15:59:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 15:59:56.747982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b296e-177f-4ac3-b742-f5a6bdcc7c62 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:56.748520 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b296e-177f-4ac3-b742-f5a6bdcc7c62 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:56.748520 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b296e-177f-4ac3-b742-f5a6bdcc7c62 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:56.749890 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 657/1321] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:59:56.755390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e464c8-58ec-4a04-b975-2f8a5f0ea88c None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:56.755862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e464c8-58ec-4a04-b975-2f8a5f0ea88c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:56.755951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e464c8-58ec-4a04-b975-2f8a5f0ea88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:56.825182 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-a2e464c8-58ec-4a04-b975-2f8a5f0ea88c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:59:56.826403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a2e464c8-58ec-4a04-b975-2f8a5f0ea88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:56.891239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:59:56.933114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 665/1322] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 15:59:57.212374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b296e-177f-4ac3-b742-f5a6bdcc7c62 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 15:59:57.214005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e23b296e-177f-4ac3-b742-f5a6bdcc7c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:57.235211 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:59:57.321738 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b296e-177f-4ac3-b742-f5a6bdcc7c62 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 15:59:57.400171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd839d9-5eca-4376-ba0a-c7cbe999c096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 15:59:57.400777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd839d9-5eca-4376-ba0a-c7cbe999c096 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:57.401049 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd839d9-5eca-4376-ba0a-c7cbe999c096 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:57.401313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd839d9-5eca-4376-ba0a-c7cbe999c096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:57.402913 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-fdd839d9-5eca-4376-ba0a-c7cbe999c096 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:57.477663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 15:59:57.515902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd839d9-5eca-4376-ba0a-c7cbe999c096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 15:59:57.517353 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd839d9-5eca-4376-ba0a-c7cbe999c096 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 15:59:57.536563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 658/1323] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 15:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 15:59:59.701622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ff1b2-eae3-443e-b0b1-0327508c96c0 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:59.702209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ff1b2-eae3-443e-b0b1-0327508c96c0 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:59.702209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ff1b2-eae3-443e-b0b1-0327508c96c0 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:59.703914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 666/1324] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 15:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 15:59:59.709367 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac783ac-9240-4f92-8a9d-8de40bcd32db None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 15:59:59.709801 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac783ac-9240-4f92-8a9d-8de40bcd32db None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 15:59:59.709893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac783ac-9240-4f92-8a9d-8de40bcd32db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 15:59:59.777336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-1ac783ac-9240-4f92-8a9d-8de40bcd32db None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 15:59:59.778547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1ac783ac-9240-4f92-8a9d-8de40bcd32db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 15:59:59.889571 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 659/1325] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 15:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:00.178071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fea99c-1634-45e9-860e-9d0d2467a37a None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:00.178303 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fea99c-1634-45e9-860e-9d0d2467a37a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:00.178606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fea99c-1634-45e9-860e-9d0d2467a37a None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:00.180198 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 667/1326] 10.222.0.27 () {54 vars in 965 bytes} [Thu Sep 8 16:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:00.188024 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95de7dc-0818-4b59-93b0-df4dadc6c36d None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:00.188395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95de7dc-0818-4b59-93b0-df4dadc6c36d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:00.188735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95de7dc-0818-4b59-93b0-df4dadc6c36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:00.223117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:00.268483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-e95de7dc-0818-4b59-93b0-df4dadc6c36d None None] MFA Rules not processed for user `3aae65f9ec394f97aeafad43932dc49c`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:00.269747 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e95de7dc-0818-4b59-93b0-df4dadc6c36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:00.345729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 660/1327] 10.222.0.27 () {58 vars in 1062 bytes} [Thu Sep 8 16:00:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:00.354982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fea99c-1634-45e9-860e-9d0d2467a37a None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:00:00.356634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f7fea99c-1634-45e9-860e-9d0d2467a37a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:00.462747 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fea99c-1634-45e9-860e-9d0d2467a37a None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:00:00.538669 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1878d-01f5-474d-8603-624a6df84030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:00:00.539250 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1878d-01f5-474d-8603-624a6df84030 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:00.539570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1878d-01f5-474d-8603-624a6df84030 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:00.539836 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1878d-01f5-474d-8603-624a6df84030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:00.541422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-76b1878d-01f5-474d-8603-624a6df84030 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:00.559478 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:00.655494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1878d-01f5-474d-8603-624a6df84030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:00:00.656975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1878d-01f5-474d-8603-624a6df84030 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:00:00.675673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 668/1328] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:00:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:00:03.079053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3eb88-658a-4fd6-94db-e92a447db04b None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:03.079707 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3eb88-658a-4fd6-94db-e92a447db04b None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:03.079707 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f3eb88-658a-4fd6-94db-e92a447db04b None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:03.081073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 661/1329] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:03.086734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630e7b0-317f-4ace-90f1-a798e2e6a713 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:03.087247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630e7b0-317f-4ace-90f1-a798e2e6a713 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:03.087247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630e7b0-317f-4ace-90f1-a798e2e6a713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:03.160622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f630e7b0-317f-4ace-90f1-a798e2e6a713 None None] MFA Rules not processed for user `8ee93dd82ab0424e9e7c9bdcc0d19a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:03.161921 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f630e7b0-317f-4ace-90f1-a798e2e6a713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:03.231879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:03.265609 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 669/1330] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:00:03 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:03.540749 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3eb88-658a-4fd6-94db-e92a447db04b None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:00:03.542455 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-96f3eb88-658a-4fd6-94db-e92a447db04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:03.651971 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f3eb88-658a-4fd6-94db-e92a447db04b None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:00:03.673219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:03.729470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445b487-f51a-4e43-8e7a-be3c48f4e440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:00:03.730080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445b487-f51a-4e43-8e7a-be3c48f4e440 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:03.730353 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445b487-f51a-4e43-8e7a-be3c48f4e440 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:03.730617 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445b487-f51a-4e43-8e7a-be3c48f4e440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:03.732347 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9445b487-f51a-4e43-8e7a-be3c48f4e440 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:03.844082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445b487-f51a-4e43-8e7a-be3c48f4e440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:00:03.845429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445b487-f51a-4e43-8e7a-be3c48f4e440 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:00:03.864450 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 662/1331] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:00:03 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:00:06.398032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301746c4-7c36-4144-b374-331f16fd3118 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:06.398860 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301746c4-7c36-4144-b374-331f16fd3118 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:06.398860 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301746c4-7c36-4144-b374-331f16fd3118 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:06.401077 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 670/1332] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:00:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:06.406665 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251783b-0e00-47eb-8bd7-22993ad4ff60 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:06.407113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251783b-0e00-47eb-8bd7-22993ad4ff60 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:06.407215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0251783b-0e00-47eb-8bd7-22993ad4ff60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:06.484503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:06.487671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-0251783b-0e00-47eb-8bd7-22993ad4ff60 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:06.489301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0251783b-0e00-47eb-8bd7-22993ad4ff60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:06.591919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 663/1333] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:00:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:06.872052 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301746c4-7c36-4144-b374-331f16fd3118 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:00:06.873617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-301746c4-7c36-4144-b374-331f16fd3118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:06.994394 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301746c4-7c36-4144-b374-331f16fd3118 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:00:07.069537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08128381-6a0a-4349-964f-e1aa5b634845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:00:07.070143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08128381-6a0a-4349-964f-e1aa5b634845 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:07.070414 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08128381-6a0a-4349-964f-e1aa5b634845 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:07.070678 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08128381-6a0a-4349-964f-e1aa5b634845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:07.072464 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-08128381-6a0a-4349-964f-e1aa5b634845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:07.099800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:07.196862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08128381-6a0a-4349-964f-e1aa5b634845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:00:07.198393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08128381-6a0a-4349-964f-e1aa5b634845 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:00:07.223598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 671/1334] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:00:09.406556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be5e80-9071-462c-a801-87d06566fd74 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:09.407502 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be5e80-9071-462c-a801-87d06566fd74 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:09.407502 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be5e80-9071-462c-a801-87d06566fd74 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:09.409659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 664/1335] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:00:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:09.417617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109a090-e358-49ba-9114-b33ff65bac86 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:09.418060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109a090-e358-49ba-9114-b33ff65bac86 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:09.418156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109a090-e358-49ba-9114-b33ff65bac86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:09.489203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-c109a090-e358-49ba-9114-b33ff65bac86 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:09.490359 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c109a090-e358-49ba-9114-b33ff65bac86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:09.570543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:09.594337 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 672/1336] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:00:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:00:09.882558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be5e80-9071-462c-a801-87d06566fd74 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:00:09.885135 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-15be5e80-9071-462c-a801-87d06566fd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:10.001546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be5e80-9071-462c-a801-87d06566fd74 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:00:10.034290 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:10.080746 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdf2143-004b-4cd8-951a-5eac6f902c2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:00:10.081349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf2143-004b-4cd8-951a-5eac6f902c2b service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:10.081621 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf2143-004b-4cd8-951a-5eac6f902c2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:10.081888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf2143-004b-4cd8-951a-5eac6f902c2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:10.083474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-dcdf2143-004b-4cd8-951a-5eac6f902c2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:10.197188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf2143-004b-4cd8-951a-5eac6f902c2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:00:10.198623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdf2143-004b-4cd8-951a-5eac6f902c2b service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:00:10.218746 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 665/1337] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:00:12.857292 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a2b07-f74c-4d5c-b19d-e84509fc9c14 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:12.857931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a2b07-f74c-4d5c-b19d-e84509fc9c14 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:12.857931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a2b07-f74c-4d5c-b19d-e84509fc9c14 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:12.859208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 673/1338] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:12.865699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc201121-d89d-4d82-b2ce-5e20728c4075 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:12.866485 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc201121-d89d-4d82-b2ce-5e20728c4075 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:12.866587 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc201121-d89d-4d82-b2ce-5e20728c4075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:12.936227 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-dc201121-d89d-4d82-b2ce-5e20728c4075 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:12.937450 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-dc201121-d89d-4d82-b2ce-5e20728c4075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:12.946004 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:13.036333 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 666/1339] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:00:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:13.323830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a2b07-f74c-4d5c-b19d-e84509fc9c14 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:00:13.325457 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-970a2b07-f74c-4d5c-b19d-e84509fc9c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:13.445142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a2b07-f74c-4d5c-b19d-e84509fc9c14 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:00:13.522476 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a5ee2-ca15-43ea-af29-c22727bfa84c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:00:13.523110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a5ee2-ca15-43ea-af29-c22727bfa84c service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:13.523380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a5ee2-ca15-43ea-af29-c22727bfa84c service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:13.523675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a5ee2-ca15-43ea-af29-c22727bfa84c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:13.525273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-df2a5ee2-ca15-43ea-af29-c22727bfa84c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:13.569798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:13.650171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a5ee2-ca15-43ea-af29-c22727bfa84c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:00:13.651667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a5ee2-ca15-43ea-af29-c22727bfa84c service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:00:13.670565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 674/1340] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:00:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:00:17.800042 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08226b51-37c2-4123-ad6c-6eafbecf1ded None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:17.800581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08226b51-37c2-4123-ad6c-6eafbecf1ded None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:17.800581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08226b51-37c2-4123-ad6c-6eafbecf1ded None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:17.801967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 667/1341] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:17.807612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061ac7-387e-4551-a9fe-9c07d47a8c12 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:17.808053 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061ac7-387e-4551-a9fe-9c07d47a8c12 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:17.808269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061ac7-387e-4551-a9fe-9c07d47a8c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:17.875796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-a9061ac7-387e-4551-a9fe-9c07d47a8c12 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:17.876902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a9061ac7-387e-4551-a9fe-9c07d47a8c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:17.964914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:17.977142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 675/1342] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:00:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:18.254385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08226b51-37c2-4123-ad6c-6eafbecf1ded None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:00:18.255987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-08226b51-37c2-4123-ad6c-6eafbecf1ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:18.366473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08226b51-37c2-4123-ad6c-6eafbecf1ded None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:00:18.405497 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:18.443916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622f0c9-2c2d-4b73-bbe2-4937bedecbb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:00:18.445074 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622f0c9-2c2d-4b73-bbe2-4937bedecbb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:18.445074 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622f0c9-2c2d-4b73-bbe2-4937bedecbb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:18.445074 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622f0c9-2c2d-4b73-bbe2-4937bedecbb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:18.447946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3622f0c9-2c2d-4b73-bbe2-4937bedecbb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:18.561228 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622f0c9-2c2d-4b73-bbe2-4937bedecbb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:00:18.562677 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622f0c9-2c2d-4b73-bbe2-4937bedecbb7 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:00:18.581963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 668/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:00:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:00:21.344118 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dad71-3612-46a9-bfe4-1120137c2a61 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:21.344712 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dad71-3612-46a9-bfe4-1120137c2a61 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:21.344712 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dad71-3612-46a9-bfe4-1120137c2a61 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:21.346002 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 676/1344] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:21.351422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13b6f0-3c09-4ec9-9419-8195c35d258d None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:21.351892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13b6f0-3c09-4ec9-9419-8195c35d258d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:21.351986 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e13b6f0-3c09-4ec9-9419-8195c35d258d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:21.420902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-6e13b6f0-3c09-4ec9-9419-8195c35d258d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:21.422125 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6e13b6f0-3c09-4ec9-9419-8195c35d258d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:21.441385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:21.524248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 669/1345] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:00:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:21.807676 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8dad71-3612-46a9-bfe4-1120137c2a61 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:00:21.809261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6c8dad71-3612-46a9-bfe4-1120137c2a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:21.922231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8dad71-3612-46a9-bfe4-1120137c2a61 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:00:21.999466 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86011146-c033-468e-97df-feb359fa4ec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:00:22.000086 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86011146-c033-468e-97df-feb359fa4ec9 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:22.000359 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86011146-c033-468e-97df-feb359fa4ec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:22.000617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86011146-c033-468e-97df-feb359fa4ec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:22.002193 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-86011146-c033-468e-97df-feb359fa4ec9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:22.046345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:22.115118 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86011146-c033-468e-97df-feb359fa4ec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:00:22.116602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86011146-c033-468e-97df-feb359fa4ec9 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:00:22.135132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 677/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:00:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:00:26.249695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e6fe9-c046-4bd3-b7a4-59c328bb3973 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:26.250182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e6fe9-c046-4bd3-b7a4-59c328bb3973 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:26.250278 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e6fe9-c046-4bd3-b7a4-59c328bb3973 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:26.257923 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5d920-d348-49ed-af03-2c5e80ef31cb None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:26.258336 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5d920-d348-49ed-af03-2c5e80ef31cb None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:26.258530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5d920-d348-49ed-af03-2c5e80ef31cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:26.259570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 670/1347] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:00:26 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:26.344036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-6df5d920-d348-49ed-af03-2c5e80ef31cb None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:26.345350 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6df5d920-d348-49ed-af03-2c5e80ef31cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:26.452581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:26.454847 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 678/1348] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:26.741768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e6fe9-c046-4bd3-b7a4-59c328bb3973 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:00:26.744403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-fb8e6fe9-c046-4bd3-b7a4-59c328bb3973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:26.858797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e6fe9-c046-4bd3-b7a4-59c328bb3973 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:00:26.909935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:26.938001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e991e-7765-4d6c-a2bc-e89a66cc35b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:00:26.938627 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e991e-7765-4d6c-a2bc-e89a66cc35b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:26.938928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e991e-7765-4d6c-a2bc-e89a66cc35b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:26.939186 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e991e-7765-4d6c-a2bc-e89a66cc35b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:26.940819 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-163e991e-7765-4d6c-a2bc-e89a66cc35b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:27.057156 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e991e-7765-4d6c-a2bc-e89a66cc35b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:00:27.058594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163e991e-7765-4d6c-a2bc-e89a66cc35b3 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:00:27.084242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 671/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:00:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:00:31.954522 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f603d4-42a5-47a2-bd93-e52b7482dbcc None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:31.954995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f603d4-42a5-47a2-bd93-e52b7482dbcc None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:31.955080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f603d4-42a5-47a2-bd93-e52b7482dbcc None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:31.956976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 679/1350] 10.222.0.27 () {54 vars in 921 bytes} [Thu Sep 8 16:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:31.962839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023c42a-c75b-4421-ad5f-5de367607953 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:31.963276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023c42a-c75b-4421-ad5f-5de367607953 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:31.963423 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5023c42a-c75b-4421-ad5f-5de367607953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:32.104694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-5023c42a-c75b-4421-ad5f-5de367607953 None None] MFA Rules not processed for user `dd28bb31b55e45668212c9ab69ed3ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:32.106143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5023c42a-c75b-4421-ad5f-5de367607953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:32.143558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:32.281979 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 672/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Sep 8 16:00:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:36.467863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce4d0b-5248-45e5-98ae-271a2afe360a None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:36.468413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce4d0b-5248-45e5-98ae-271a2afe360a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:36.468413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce4d0b-5248-45e5-98ae-271a2afe360a None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:36.469811 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 680/1352] 10.222.0.27 () {54 vars in 921 bytes} [Thu Sep 8 16:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:36.474682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d046294-24f8-48a3-aa20-661f7a80c294 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:36.475159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d046294-24f8-48a3-aa20-661f7a80c294 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:36.475542 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d046294-24f8-48a3-aa20-661f7a80c294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:36.556895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-4d046294-24f8-48a3-aa20-661f7a80c294 None None] MFA Rules not processed for user `dd28bb31b55e45668212c9ab69ed3ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:36.558337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4d046294-24f8-48a3-aa20-661f7a80c294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:36.635359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:36.675771 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 673/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Sep 8 16:00:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:38.202897 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49097081-1b53-4d5d-a455-00b6fec32131 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:38.203492 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49097081-1b53-4d5d-a455-00b6fec32131 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:38.203492 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49097081-1b53-4d5d-a455-00b6fec32131 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:38.204787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 681/1354] 10.222.0.27 () {54 vars in 921 bytes} [Thu Sep 8 16:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:38.209948 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3f4bb-9378-4de1-b00e-d6598ebf4b94 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:38.210389 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3f4bb-9378-4de1-b00e-d6598ebf4b94 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:38.210513 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3f4bb-9378-4de1-b00e-d6598ebf4b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:38.292370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-a3b3f4bb-9378-4de1-b00e-d6598ebf4b94 None None] MFA Rules not processed for user `dd28bb31b55e45668212c9ab69ed3ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:38.293718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a3b3f4bb-9378-4de1-b00e-d6598ebf4b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:38.416001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 674/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Sep 8 16:00:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:00:41.795910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf32c3-cb50-4698-bc8a-66c29739f63f None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:41.796873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf32c3-cb50-4698-bc8a-66c29739f63f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:41.797354 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf32c3-cb50-4698-bc8a-66c29739f63f None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:41.799090 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 682/1356] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:00:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:41.804936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b09433-58ef-4d04-80c3-cfaf578c5a34 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:41.806343 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b09433-58ef-4d04-80c3-cfaf578c5a34 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:41.806830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b09433-58ef-4d04-80c3-cfaf578c5a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:41.848549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:41.882679 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-59b09433-58ef-4d04-80c3-cfaf578c5a34 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:41.883903 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-59b09433-58ef-4d04-80c3-cfaf578c5a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:41.984247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 675/1357] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:00:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:42.287858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90026e-2603-45ea-b31f-5af9b06b6d87 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:42.289151 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90026e-2603-45ea-b31f-5af9b06b6d87 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:42.289701 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90026e-2603-45ea-b31f-5af9b06b6d87 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:42.293668 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 683/1358] 10.222.0.27 () {54 vars in 962 bytes} [Thu Sep 8 16:00:42 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:42.301489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ad284-c12f-4fe6-afd2-fafaa61373c0 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:42.302035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ad284-c12f-4fe6-afd2-fafaa61373c0 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:42.302430 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ad284-c12f-4fe6-afd2-fafaa61373c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:42.394239 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-159ad284-c12f-4fe6-afd2-fafaa61373c0 None None] MFA Rules not processed for user `6e9209e77ce44b128e25a8ddcf78f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:42.395635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-159ad284-c12f-4fe6-afd2-fafaa61373c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:42.445922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:42.535267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 676/1359] 10.222.0.27 () {58 vars in 1059 bytes} [Thu Sep 8 16:00:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:00:42.543709 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90026e-2603-45ea-b31f-5af9b06b6d87 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:00:42.545449 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-fe90026e-2603-45ea-b31f-5af9b06b6d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:42.699507 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90026e-2603-45ea-b31f-5af9b06b6d87 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:00:42.778617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:42.821134 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d28020-aa2f-4f7d-b65e-124e3def8122 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:00:42.821780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d28020-aa2f-4f7d-b65e-124e3def8122 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:42.822033 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d28020-aa2f-4f7d-b65e-124e3def8122 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:42.822279 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d28020-aa2f-4f7d-b65e-124e3def8122 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:42.823910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-60d28020-aa2f-4f7d-b65e-124e3def8122 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:42.930091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d28020-aa2f-4f7d-b65e-124e3def8122 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:00:42.931770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d28020-aa2f-4f7d-b65e-124e3def8122 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:00:42.950611 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 684/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:00:45.306576 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ddb2f-94f1-4ebd-9c25-beabce234c5a None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:45.307018 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ddb2f-94f1-4ebd-9c25-beabce234c5a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:45.307209 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ddb2f-94f1-4ebd-9c25-beabce234c5a None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:45.308917 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 677/1361] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:45.315137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170db52-8c52-4676-8901-7fc756b6c3d6 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:45.315602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170db52-8c52-4676-8901-7fc756b6c3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:45.315804 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4170db52-8c52-4676-8901-7fc756b6c3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:45.383034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-4170db52-8c52-4676-8901-7fc756b6c3d6 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:45.384256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-4170db52-8c52-4676-8901-7fc756b6c3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:45.423393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:45.490097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 685/1362] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:00:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:45.782727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ddb2f-94f1-4ebd-9c25-beabce234c5a None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:00:45.784349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e29ddb2f-94f1-4ebd-9c25-beabce234c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:45.974200 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ddb2f-94f1-4ebd-9c25-beabce234c5a None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:00:45.986076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:46.124931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197d1bb-1af6-4a02-8779-6b87c78f340b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:00:46.125549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197d1bb-1af6-4a02-8779-6b87c78f340b service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:46.125827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197d1bb-1af6-4a02-8779-6b87c78f340b service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:46.126100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197d1bb-1af6-4a02-8779-6b87c78f340b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:46.127736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5197d1bb-1af6-4a02-8779-6b87c78f340b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:46.221160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197d1bb-1af6-4a02-8779-6b87c78f340b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:00:46.222838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197d1bb-1af6-4a02-8779-6b87c78f340b service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:00:46.242915 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 678/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:00:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:00:48.694019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153cdb0-acae-4afa-b397-22ebaea8be0f None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:48.694635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153cdb0-acae-4afa-b397-22ebaea8be0f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:48.694635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153cdb0-acae-4afa-b397-22ebaea8be0f None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:48.696185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 686/1364] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:48.702057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc16a8-62fe-41a4-9539-dd98818a0e00 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:48.702526 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc16a8-62fe-41a4-9539-dd98818a0e00 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:48.702636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bc16a8-62fe-41a4-9539-dd98818a0e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:48.719130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:48.773532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-78bc16a8-62fe-41a4-9539-dd98818a0e00 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:48.774869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-78bc16a8-62fe-41a4-9539-dd98818a0e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:48.879674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 679/1365] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:00:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:49.163536 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884cc66-05a4-4e09-ac88-559c4a23174e None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:49.163814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884cc66-05a4-4e09-ac88-559c4a23174e None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:49.164282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1884cc66-05a4-4e09-ac88-559c4a23174e None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:49.165876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 687/1366] 10.222.0.27 () {54 vars in 962 bytes} [Thu Sep 8 16:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:49.177387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcc0c0-760b-4389-946d-ed276f4c57f6 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:49.177662 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcc0c0-760b-4389-946d-ed276f4c57f6 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:49.177931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfcc0c0-760b-4389-946d-ed276f4c57f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:49.258922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-7dfcc0c0-760b-4389-946d-ed276f4c57f6 None None] MFA Rules not processed for user `6e9209e77ce44b128e25a8ddcf78f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:49.260264 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-7dfcc0c0-760b-4389-946d-ed276f4c57f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:49.296982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:49.380711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 680/1367] 10.222.0.27 () {58 vars in 1059 bytes} [Thu Sep 8 16:00:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:49.388915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884cc66-05a4-4e09-ac88-559c4a23174e None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:00:49.390700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-1884cc66-05a4-4e09-ac88-559c4a23174e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:49.552172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1884cc66-05a4-4e09-ac88-559c4a23174e None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:00:49.573887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:49.679812 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80790e3c-868b-47bd-998f-0e51f1189073 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:00:49.680294 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80790e3c-868b-47bd-998f-0e51f1189073 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:49.680570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80790e3c-868b-47bd-998f-0e51f1189073 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:49.680845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80790e3c-868b-47bd-998f-0e51f1189073 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:49.682503 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-80790e3c-868b-47bd-998f-0e51f1189073 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:49.778920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80790e3c-868b-47bd-998f-0e51f1189073 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:00:49.780712 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80790e3c-868b-47bd-998f-0e51f1189073 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:00:49.800476 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 688/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:00:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:00:49.949461 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf9b96-4848-4569-8a3e-ba64ee7db73c None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:49.950092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf9b96-4848-4569-8a3e-ba64ee7db73c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:49.950092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf9b96-4848-4569-8a3e-ba64ee7db73c None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:49.955550 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 681/1369] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 16:00:49.957818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66876e-a8e1-40ba-8a17-5e8ce4fb6ad6 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:49.958336 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66876e-a8e1-40ba-8a17-5e8ce4fb6ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:49.958711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66876e-a8e1-40ba-8a17-5e8ce4fb6ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:49.996989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:50.041373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-6f66876e-a8e1-40ba-8a17-5e8ce4fb6ad6 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:50.043065 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6f66876e-a8e1-40ba-8a17-5e8ce4fb6ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:50.158396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 689/1370] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:00:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:50.489991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ae3e7-e82f-4ec2-91f9-20300863d562 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:50.489991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ae3e7-e82f-4ec2-91f9-20300863d562 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:50.489991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ae3e7-e82f-4ec2-91f9-20300863d562 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:50.492234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 682/1371] 10.222.0.27 () {54 vars in 964 bytes} [Thu Sep 8 16:00:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:50.497426 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d94d9-d6ed-40d1-b9aa-433dec357794 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:50.497844 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d94d9-d6ed-40d1-b9aa-433dec357794 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:50.498349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d94d9-d6ed-40d1-b9aa-433dec357794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:50.582726 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-696d94d9-d6ed-40d1-b9aa-433dec357794 None None] MFA Rules not processed for user `89df988d73ff48f3b7a92d335df9b633`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:50.584269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-696d94d9-d6ed-40d1-b9aa-433dec357794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:50.638923 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:50.671739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 690/1372] 10.222.0.27 () {58 vars in 1061 bytes} [Thu Sep 8 16:00:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:50.681541 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ae3e7-e82f-4ec2-91f9-20300863d562 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:00:50.683193 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f45ae3e7-e82f-4ec2-91f9-20300863d562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:50.797398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ae3e7-e82f-4ec2-91f9-20300863d562 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:00:50.848384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:50.886590 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81913afd-42a3-4101-88b7-8f35a01543c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:00:50.887239 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81913afd-42a3-4101-88b7-8f35a01543c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:50.887546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81913afd-42a3-4101-88b7-8f35a01543c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:50.887811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81913afd-42a3-4101-88b7-8f35a01543c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:50.889436 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-81913afd-42a3-4101-88b7-8f35a01543c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:51.032550 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81913afd-42a3-4101-88b7-8f35a01543c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:00:51.034399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81913afd-42a3-4101-88b7-8f35a01543c6 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:00:51.057447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 683/1373] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:00:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:00:51.993878 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58985017-3314-4980-a7e8-626931ab69dc None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:51.994431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58985017-3314-4980-a7e8-626931ab69dc None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:51.994431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58985017-3314-4980-a7e8-626931ab69dc None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:51.996014 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 691/1374] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:52.001215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ab0a0-de91-42d2-9840-c9d35490f87d None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:52.001546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ab0a0-de91-42d2-9840-c9d35490f87d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:52.001804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ab0a0-de91-42d2-9840-c9d35490f87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:52.067927 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-1c5ab0a0-de91-42d2-9840-c9d35490f87d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:52.069128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1c5ab0a0-de91-42d2-9840-c9d35490f87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:52.090669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:52.166466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 684/1375] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:00:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:52.453178 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58985017-3314-4980-a7e8-626931ab69dc None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:00:52.454866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-58985017-3314-4980-a7e8-626931ab69dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:52.628993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58985017-3314-4980-a7e8-626931ab69dc None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:00:52.688886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:52.754069 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781eb34-2af9-4537-b8b2-b98c22b2abb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:00:52.754674 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781eb34-2af9-4537-b8b2-b98c22b2abb3 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:52.754931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781eb34-2af9-4537-b8b2-b98c22b2abb3 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:52.755202 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781eb34-2af9-4537-b8b2-b98c22b2abb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:52.756822 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e781eb34-2af9-4537-b8b2-b98c22b2abb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:52.848307 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781eb34-2af9-4537-b8b2-b98c22b2abb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:00:52.850148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781eb34-2af9-4537-b8b2-b98c22b2abb3 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:00:52.868238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 692/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:00:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:00:55.077766 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36374f-2b05-4055-9111-98e66627abf6 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:55.078336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36374f-2b05-4055-9111-98e66627abf6 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:55.078420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c36374f-2b05-4055-9111-98e66627abf6 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:55.080026 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 685/1377] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:00:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:55.085752 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e3cb6-4c8c-4476-a100-6d4d1a9afc2b None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:55.086221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e3cb6-4c8c-4476-a100-6d4d1a9afc2b None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:55.086328 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e3cb6-4c8c-4476-a100-6d4d1a9afc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:55.155245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-2a0e3cb6-4c8c-4476-a100-6d4d1a9afc2b None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:55.156491 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2a0e3cb6-4c8c-4476-a100-6d4d1a9afc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:55.164637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:55.267738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 693/1378] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:55.556983 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36374f-2b05-4055-9111-98e66627abf6 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:00:55.559216 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6c36374f-2b05-4055-9111-98e66627abf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:55.725240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c36374f-2b05-4055-9111-98e66627abf6 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:00:55.729675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:55.853684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc4811-e668-4eb7-9310-1991cc18c6be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:00:55.854202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc4811-e668-4eb7-9310-1991cc18c6be service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:55.854470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc4811-e668-4eb7-9310-1991cc18c6be service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:55.854728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc4811-e668-4eb7-9310-1991cc18c6be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:55.856369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4cdc4811-e668-4eb7-9310-1991cc18c6be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:55.951744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc4811-e668-4eb7-9310-1991cc18c6be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:00:55.953436 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdc4811-e668-4eb7-9310-1991cc18c6be service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:00:55.971999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 686/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:00:58.156625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705687bd-10c0-4ec4-ab07-7fbd8ebcf6e5 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:58.157238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705687bd-10c0-4ec4-ab07-7fbd8ebcf6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:58.157238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705687bd-10c0-4ec4-ab07-7fbd8ebcf6e5 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:58.158490 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 694/1380] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:00:58.163854 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27d637-0c07-4ec6-a322-620105c2308d None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:58.164318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27d637-0c07-4ec6-a322-620105c2308d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:58.164400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f27d637-0c07-4ec6-a322-620105c2308d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:58.176947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:58.241931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-7f27d637-0c07-4ec6-a322-620105c2308d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:00:58.243135 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-7f27d637-0c07-4ec6-a322-620105c2308d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:58.348998 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 687/1381] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:00:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:00:58.636561 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705687bd-10c0-4ec4-ab07-7fbd8ebcf6e5 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:00:58.639071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-705687bd-10c0-4ec4-ab07-7fbd8ebcf6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:58.787282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:58.800751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705687bd-10c0-4ec4-ab07-7fbd8ebcf6e5 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:00:58.930583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153faca-6ff7-40f1-824d-dd37b5865005 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:00:58.931270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153faca-6ff7-40f1-824d-dd37b5865005 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:00:58.931603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153faca-6ff7-40f1-824d-dd37b5865005 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:00:58.931873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153faca-6ff7-40f1-824d-dd37b5865005 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:00:58.933998 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f153faca-6ff7-40f1-824d-dd37b5865005 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:00:59.042651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153faca-6ff7-40f1-824d-dd37b5865005 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:00:59.044908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153faca-6ff7-40f1-824d-dd37b5865005 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:00:59.059319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:00:59.070194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 695/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:00:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:01.265962 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c8182-98c9-4416-8ae4-6bde88641696 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:01.266630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c8182-98c9-4416-8ae4-6bde88641696 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:01.266630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8c8182-98c9-4416-8ae4-6bde88641696 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:01.268044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 688/1383] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:01.274825 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946c615-b78c-4f3e-acd3-dadd2e7913bf None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:01.275949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946c615-b78c-4f3e-acd3-dadd2e7913bf None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:01.276377 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946c615-b78c-4f3e-acd3-dadd2e7913bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:01.357376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-0946c615-b78c-4f3e-acd3-dadd2e7913bf None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:01.358661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0946c615-b78c-4f3e-acd3-dadd2e7913bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:01.462967 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 696/1384] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:01.755041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c8182-98c9-4416-8ae4-6bde88641696 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:01.756745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-cf8c8182-98c9-4416-8ae4-6bde88641696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:01.865256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:01.931136 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8c8182-98c9-4416-8ae4-6bde88641696 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:02.066331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae1bc9-4c08-4626-abf7-9876498207f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:02.067001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae1bc9-4c08-4626-abf7-9876498207f2 service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:02.067305 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae1bc9-4c08-4626-abf7-9876498207f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:02.067551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae1bc9-4c08-4626-abf7-9876498207f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:02.069239 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d8ae1bc9-4c08-4626-abf7-9876498207f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:02.124593 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:02.181806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae1bc9-4c08-4626-abf7-9876498207f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:02.184059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae1bc9-4c08-4626-abf7-9876498207f2 service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:02.215316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 689/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:04.580755 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e059d-a7f2-43a0-984e-2f9f038d09b9 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:04.581411 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e059d-a7f2-43a0-984e-2f9f038d09b9 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:04.581411 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9e059d-a7f2-43a0-984e-2f9f038d09b9 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:04.582628 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 697/1386] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:04.588193 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8741fc-64e5-40a2-a909-f9b2bec801ac None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:04.588694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8741fc-64e5-40a2-a909-f9b2bec801ac None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:04.588789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8741fc-64e5-40a2-a909-f9b2bec801ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:04.658262 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-cd8741fc-64e5-40a2-a909-f9b2bec801ac None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:04.659488 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-cd8741fc-64e5-40a2-a909-f9b2bec801ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:04.763693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 690/1387] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:05.047944 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e059d-a7f2-43a0-984e-2f9f038d09b9 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:05.049468 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ab9e059d-a7f2-43a0-984e-2f9f038d09b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:05.126389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:05.201192 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9e059d-a7f2-43a0-984e-2f9f038d09b9 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:05.317269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb304ffa-e87c-44e4-8831-2220d5ac24ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:05.317792 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb304ffa-e87c-44e4-8831-2220d5ac24ff service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:05.318062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb304ffa-e87c-44e4-8831-2220d5ac24ff service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:05.318327 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb304ffa-e87c-44e4-8831-2220d5ac24ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:05.319976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-eb304ffa-e87c-44e4-8831-2220d5ac24ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:05.360430 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:05.413987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb304ffa-e87c-44e4-8831-2220d5ac24ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:05.415633 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb304ffa-e87c-44e4-8831-2220d5ac24ff service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:05.433558 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 698/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:06.238563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b342ce4-7737-4092-8ea9-bcc19f94c710 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:06.239128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b342ce4-7737-4092-8ea9-bcc19f94c710 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:06.239222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b342ce4-7737-4092-8ea9-bcc19f94c710 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:06.240671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 691/1389] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:06.246318 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088e270-98bd-4405-ae3b-f864c9883a09 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:06.246766 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088e270-98bd-4405-ae3b-f864c9883a09 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:06.246766 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088e270-98bd-4405-ae3b-f864c9883a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:06.313487 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-5088e270-98bd-4405-ae3b-f864c9883a09 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:06.314724 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5088e270-98bd-4405-ae3b-f864c9883a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:06.417679 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 699/1390] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:01:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:01:06.713841 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1859134-a69f-4f19-9ce6-0c231c487f59 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:06.714143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1859134-a69f-4f19-9ce6-0c231c487f59 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:06.714408 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1859134-a69f-4f19-9ce6-0c231c487f59 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:06.716013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 692/1391] 10.222.0.27 () {54 vars in 966 bytes} [Thu Sep 8 16:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:06.721360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9555daa-69ed-4a19-9b06-a96e1493eaa8 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:06.721646 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9555daa-69ed-4a19-9b06-a96e1493eaa8 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:06.721912 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9555daa-69ed-4a19-9b06-a96e1493eaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:06.738748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:06.805429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f9555daa-69ed-4a19-9b06-a96e1493eaa8 None None] MFA Rules not processed for user `f24ee16c043244938901c4e850dea967`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:06.806530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f9555daa-69ed-4a19-9b06-a96e1493eaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:06.885201 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 700/1392] 10.222.0.27 () {58 vars in 1063 bytes} [Thu Sep 8 16:01:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:06.894235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1859134-a69f-4f19-9ce6-0c231c487f59 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:06.895785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d1859134-a69f-4f19-9ce6-0c231c487f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:06.929882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:07.005284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1859134-a69f-4f19-9ce6-0c231c487f59 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:07.082064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96565759-bd5f-46cc-8ecc-2ff1152982b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f24ee16c043244938901c4e850dea967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:07.082581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96565759-bd5f-46cc-8ecc-2ff1152982b2 service glance] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:07.082851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96565759-bd5f-46cc-8ecc-2ff1152982b2 service glance] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:07.083113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96565759-bd5f-46cc-8ecc-2ff1152982b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:07.084697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-96565759-bd5f-46cc-8ecc-2ff1152982b2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:07.162632 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:07.199846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96565759-bd5f-46cc-8ecc-2ff1152982b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:07.201290 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96565759-bd5f-46cc-8ecc-2ff1152982b2 service glance] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:07.219747 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 693/1393] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:07.660564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53251380-fd83-4308-9d9c-009b98dfbac4 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:07.661099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53251380-fd83-4308-9d9c-009b98dfbac4 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:07.661099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53251380-fd83-4308-9d9c-009b98dfbac4 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:07.662345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 701/1394] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:07.667919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2923a0-639f-4afe-826a-af9e1a143920 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:07.668389 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2923a0-639f-4afe-826a-af9e1a143920 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:07.668492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2923a0-639f-4afe-826a-af9e1a143920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:07.737821 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-8b2923a0-639f-4afe-826a-af9e1a143920 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:07.739260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-8b2923a0-639f-4afe-826a-af9e1a143920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:07.847505 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 694/1395] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:07.977073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b46ff-5ee8-4796-b372-b70ea6361c2b None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:07.977357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b46ff-5ee8-4796-b372-b70ea6361c2b None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:07.977678 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b46ff-5ee8-4796-b372-b70ea6361c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:08.046204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-113b46ff-5ee8-4796-b372-b70ea6361c2b None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:08.047399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-113b46ff-5ee8-4796-b372-b70ea6361c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:08.106071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:08.139526 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2923a0-639f-4afe-826a-af9e1a143920 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:08.139845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 702/1396] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Sep 8 16:01:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:08.140754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-8b2923a0-639f-4afe-826a-af9e1a143920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:08.182258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:08.188357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1780c5f-f48d-4db2-bfce-de45e8ce8faf None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:08.188642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1780c5f-f48d-4db2-bfce-de45e8ce8faf None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:08.188915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1780c5f-f48d-4db2-bfce-de45e8ce8faf None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:08.190282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 703/1397] 10.222.0.27 () {54 vars in 943 bytes} [Thu Sep 8 16:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 16:01:08.196560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3a0f2-e681-430a-aecf-23e81ce6d94e None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:08.196780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3a0f2-e681-430a-aecf-23e81ce6d94e None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:08.197048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3a0f2-e681-430a-aecf-23e81ce6d94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:08.259064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2923a0-639f-4afe-826a-af9e1a143920 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:08.276055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-0ee3a0f2-e681-430a-aecf-23e81ce6d94e None None] MFA Rules not processed for user `de54687547a14ab89cc41a1fabaab400`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:08.277952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0ee3a0f2-e681-430a-aecf-23e81ce6d94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:08.374130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa088c5-76a5-4dee-9f7a-acd441726c01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:08.374745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa088c5-76a5-4dee-9f7a-acd441726c01 service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:08.375017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa088c5-76a5-4dee-9f7a-acd441726c01 service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:08.375283 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa088c5-76a5-4dee-9f7a-acd441726c01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:08.377612 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6fa088c5-76a5-4dee-9f7a-acd441726c01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:08.387372 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 704/1398] 10.222.0.27 () {58 vars in 1040 bytes} [Thu Sep 8 16:01:08 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:08.397284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3a0f2-e681-430a-aecf-23e81ce6d94e None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:08.406937 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee3a0f2-e681-430a-aecf-23e81ce6d94e None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:08.424656 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:08.453709 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:08.501045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa088c5-76a5-4dee-9f7a-acd441726c01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:08.502663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa088c5-76a5-4dee-9f7a-acd441726c01 service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:08.520806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 695/1399] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:01:08.538762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9fdb1c-5a35-42a3-923d-49e2b6c3b725 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:08.539576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fdb1c-5a35-42a3-923d-49e2b6c3b725 service swift] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:08.539856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fdb1c-5a35-42a3-923d-49e2b6c3b725 service swift] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:08.540192 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9fdb1c-5a35-42a3-923d-49e2b6c3b725 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:08.542027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ea9fdb1c-5a35-42a3-923d-49e2b6c3b725 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:08.635893 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fdb1c-5a35-42a3-923d-49e2b6c3b725 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:08.637613 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9fdb1c-5a35-42a3-923d-49e2b6c3b725 service swift] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:08.646773 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 705/1400] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:01:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 16:01:10.717289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28170989-35ba-4f57-a737-3576baaf6ed5 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:10.718238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28170989-35ba-4f57-a737-3576baaf6ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:10.718517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28170989-35ba-4f57-a737-3576baaf6ed5 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:10.720209 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 696/1401] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:01:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:10.725955 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8e4ce-80ba-465b-98b9-7acc5bc884a6 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:10.726409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8e4ce-80ba-465b-98b9-7acc5bc884a6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:10.726504 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8e4ce-80ba-465b-98b9-7acc5bc884a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:10.799024 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-36b8e4ce-80ba-465b-98b9-7acc5bc884a6 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:10.800362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-36b8e4ce-80ba-465b-98b9-7acc5bc884a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:10.809297 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:10.911498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 706/1402] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:11.200546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28170989-35ba-4f57-a737-3576baaf6ed5 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:11.202551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-28170989-35ba-4f57-a737-3576baaf6ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:11.358865 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28170989-35ba-4f57-a737-3576baaf6ed5 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:11.372248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:11.487725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4a95b-3ab4-4f41-bd1e-740a3eabfd2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:11.488478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4a95b-3ab4-4f41-bd1e-740a3eabfd2c service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:11.488833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4a95b-3ab4-4f41-bd1e-740a3eabfd2c service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:11.489166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4a95b-3ab4-4f41-bd1e-740a3eabfd2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:11.490778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3cb4a95b-3ab4-4f41-bd1e-740a3eabfd2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:11.588366 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4a95b-3ab4-4f41-bd1e-740a3eabfd2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:11.590220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4a95b-3ab4-4f41-bd1e-740a3eabfd2c service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:11.619675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 697/1403] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:13.779616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c4dc0-041e-49e3-ac36-08709be536b0 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:13.779616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c4dc0-041e-49e3-ac36-08709be536b0 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:13.779616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c4dc0-041e-49e3-ac36-08709be536b0 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:13.782055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 707/1404] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:01:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:13.786228 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64ca35-c179-450a-9c38-f5f8c70b4312 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:13.786700 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64ca35-c179-450a-9c38-f5f8c70b4312 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:13.786797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a64ca35-c179-450a-9c38-f5f8c70b4312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:13.809551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:13.860549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-2a64ca35-c179-450a-9c38-f5f8c70b4312 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:13.861826 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-2a64ca35-c179-450a-9c38-f5f8c70b4312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:13.965981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 698/1405] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:01:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:01:14.256039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c4dc0-041e-49e3-ac36-08709be536b0 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:14.257633 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ee4c4dc0-041e-49e3-ac36-08709be536b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:14.410154 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:14.421421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4c4dc0-041e-49e3-ac36-08709be536b0 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:14.562287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc433ea6-fb3e-41bf-9194-4489ed525e86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:14.563209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc433ea6-fb3e-41bf-9194-4489ed525e86 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:14.563622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc433ea6-fb3e-41bf-9194-4489ed525e86 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:14.563981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc433ea6-fb3e-41bf-9194-4489ed525e86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:14.570677 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-dc433ea6-fb3e-41bf-9194-4489ed525e86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:14.742074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc433ea6-fb3e-41bf-9194-4489ed525e86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:14.743784 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc433ea6-fb3e-41bf-9194-4489ed525e86 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:14.760559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:14.773192 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 708/1406] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:17.041898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae8afe-d249-4edf-9f0a-7493b7052667 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:17.042495 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae8afe-d249-4edf-9f0a-7493b7052667 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:17.042495 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae8afe-d249-4edf-9f0a-7493b7052667 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:17.044053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 699/1407] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:17.049401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bbc2c-80d4-41a0-a9d5-44dc14f700b8 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:17.049824 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bbc2c-80d4-41a0-a9d5-44dc14f700b8 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:17.050079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475bbc2c-80d4-41a0-a9d5-44dc14f700b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:17.125532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-475bbc2c-80d4-41a0-a9d5-44dc14f700b8 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:17.126741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-475bbc2c-80d4-41a0-a9d5-44dc14f700b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:17.239773 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 709/1408] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:01:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:17.528274 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae8afe-d249-4edf-9f0a-7493b7052667 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:17.530431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-caae8afe-d249-4edf-9f0a-7493b7052667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:17.631645 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:17.693655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae8afe-d249-4edf-9f0a-7493b7052667 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:17.826226 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97607aa2-9efb-43ff-972a-becbbe26750b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:17.827266 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97607aa2-9efb-43ff-972a-becbbe26750b service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:17.827572 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97607aa2-9efb-43ff-972a-becbbe26750b service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:17.827839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97607aa2-9efb-43ff-972a-becbbe26750b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:17.829530 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-97607aa2-9efb-43ff-972a-becbbe26750b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:17.893636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:17.927825 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97607aa2-9efb-43ff-972a-becbbe26750b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:17.931214 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97607aa2-9efb-43ff-972a-becbbe26750b service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:17.952431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 700/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:27.238115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=67 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:27.949114 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=20 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:31.080892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588539c-cec5-43d0-bada-c08c169a87a7 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:31.081495 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588539c-cec5-43d0-bada-c08c169a87a7 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:31.081495 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588539c-cec5-43d0-bada-c08c169a87a7 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:31.083486 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 710/1410] 10.222.0.27 () {54 vars in 918 bytes} [Thu Sep 8 16:01:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:31.089380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8035dc-3ebf-4f12-9763-6d043da0313d None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:31.089817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8035dc-3ebf-4f12-9763-6d043da0313d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:31.089931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8035dc-3ebf-4f12-9763-6d043da0313d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:31.174411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-2b8035dc-3ebf-4f12-9763-6d043da0313d None None] MFA Rules not processed for user `dd28bb31b55e45668212c9ab69ed3ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:31.175661 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-2b8035dc-3ebf-4f12-9763-6d043da0313d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:31.294652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 701/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Thu Sep 8 16:01:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:31.309645 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf30afc-31fa-4b88-a20d-41713ea4a991 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:31.309977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf30afc-31fa-4b88-a20d-41713ea4a991 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:31.310244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf30afc-31fa-4b88-a20d-41713ea4a991 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:31.311627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 711/1412] 10.222.0.27 () {54 vars in 918 bytes} [Thu Sep 8 16:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:31.316388 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c9f1a-87bf-4315-bcc8-c311e6892bf7 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:31.316672 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c9f1a-87bf-4315-bcc8-c311e6892bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:31.316964 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c9f1a-87bf-4315-bcc8-c311e6892bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:31.392819 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:31.395948 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-672c9f1a-87bf-4315-bcc8-c311e6892bf7 None None] MFA Rules not processed for user `6e9209e77ce44b128e25a8ddcf78f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:31.397570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-672c9f1a-87bf-4315-bcc8-c311e6892bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:31.535679 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 702/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Thu Sep 8 16:01:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:01:34.098534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebce605-45e4-4c86-8bf4-feb4f445e8c9 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:34.099390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebce605-45e4-4c86-8bf4-feb4f445e8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:34.099390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebce605-45e4-4c86-8bf4-feb4f445e8c9 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:34.101255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 16:01:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:34.110327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da612693-54aa-4863-b112-7e7a42bb4736 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:34.110770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da612693-54aa-4863-b112-7e7a42bb4736 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:34.110881 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da612693-54aa-4863-b112-7e7a42bb4736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:34.192296 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-da612693-54aa-4863-b112-7e7a42bb4736 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:34.193531 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-da612693-54aa-4863-b112-7e7a42bb4736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:34.249378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:34.295061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 16:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:34.588204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebce605-45e4-4c86-8bf4-feb4f445e8c9 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:34.590619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-eebce605-45e4-4c86-8bf4-feb4f445e8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:34.751325 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebce605-45e4-4c86-8bf4-feb4f445e8c9 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:34.827679 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:34.869635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ac106-fd1f-4b7e-94c1-1b752957eb3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:34.870232 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ac106-fd1f-4b7e-94c1-1b752957eb3e service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:34.870502 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ac106-fd1f-4b7e-94c1-1b752957eb3e service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:34.870764 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ac106-fd1f-4b7e-94c1-1b752957eb3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:34.872374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-fd0ac106-fd1f-4b7e-94c1-1b752957eb3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:34.984551 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ac106-fd1f-4b7e-94c1-1b752957eb3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:34.986157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ac106-fd1f-4b7e-94c1-1b752957eb3e service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:35.005067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 713/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:36.137128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389634b-6e3f-4bdb-95f4-cb114cc9bfbf None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:36.137982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389634b-6e3f-4bdb-95f4-cb114cc9bfbf None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:36.137982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389634b-6e3f-4bdb-95f4-cb114cc9bfbf None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:36.140218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 16:01:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:36.147519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532f8e9-005f-45cd-8d72-a128bcd0576f None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:36.147991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532f8e9-005f-45cd-8d72-a128bcd0576f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:36.148112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9532f8e9-005f-45cd-8d72-a128bcd0576f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:36.230546 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-9532f8e9-005f-45cd-8d72-a128bcd0576f None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:36.231805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-9532f8e9-005f-45cd-8d72-a128bcd0576f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:36.259823 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:36.336981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 16:01:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:36.489200 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389634b-6e3f-4bdb-95f4-cb114cc9bfbf None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:36.491647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c389634b-6e3f-4bdb-95f4-cb114cc9bfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:36.656813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389634b-6e3f-4bdb-95f4-cb114cc9bfbf None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:36.698246 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:36.773812 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d08c09-4895-4c49-b2c1-4d3727508635 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:36.774425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d08c09-4895-4c49-b2c1-4d3727508635 service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:36.774696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d08c09-4895-4c49-b2c1-4d3727508635 service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:36.774962 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d08c09-4895-4c49-b2c1-4d3727508635 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:36.776593 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a4d08c09-4895-4c49-b2c1-4d3727508635 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:36.869502 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d08c09-4895-4c49-b2c1-4d3727508635 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:36.871108 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d08c09-4895-4c49-b2c1-4d3727508635 service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:36.889397 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 705/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:38.017008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360a891-ead2-49d7-ac2d-654ab47b73a8 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:38.017945 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360a891-ead2-49d7-ac2d-654ab47b73a8 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:38.017945 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8360a891-ead2-49d7-ac2d-654ab47b73a8 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:38.020135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 16:01:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:38.026677 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c7c65-a6da-4a93-abf5-32e95ebf2861 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:38.027127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c7c65-a6da-4a93-abf5-32e95ebf2861 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:38.027250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c7c65-a6da-4a93-abf5-32e95ebf2861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:38.085922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:38.111144 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-3c6c7c65-a6da-4a93-abf5-32e95ebf2861 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:38.112409 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3c6c7c65-a6da-4a93-abf5-32e95ebf2861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:38.218678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 16:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:01:38.369750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360a891-ead2-49d7-ac2d-654ab47b73a8 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:38.371642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8360a891-ead2-49d7-ac2d-654ab47b73a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:38.531739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8360a891-ead2-49d7-ac2d-654ab47b73a8 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:38.544113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:38.650624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d888f4c-0737-48dc-8626-95ec455158ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:38.650624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d888f4c-0737-48dc-8626-95ec455158ad service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:38.650624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d888f4c-0737-48dc-8626-95ec455158ad service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:38.650624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d888f4c-0737-48dc-8626-95ec455158ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:38.652104 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-4d888f4c-0737-48dc-8626-95ec455158ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:38.759607 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d888f4c-0737-48dc-8626-95ec455158ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:38.761232 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d888f4c-0737-48dc-8626-95ec455158ad service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:38.780525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 716/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:39.888297 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036218e-d3a4-4896-84d4-c195c6b587a8 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:39.888878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036218e-d3a4-4896-84d4-c195c6b587a8 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:39.888878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036218e-d3a4-4896-84d4-c195c6b587a8 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:39.890138 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 16:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:39.897449 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2fca3-cb2f-42b1-afc6-5cacd0b6548b None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:39.898211 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2fca3-cb2f-42b1-afc6-5cacd0b6548b None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:39.898211 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d2fca3-cb2f-42b1-afc6-5cacd0b6548b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:39.924642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:39.984125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-97d2fca3-cb2f-42b1-afc6-5cacd0b6548b None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:39.985317 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-97d2fca3-cb2f-42b1-afc6-5cacd0b6548b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:40.088512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 16:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:40.239055 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036218e-d3a4-4896-84d4-c195c6b587a8 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:40.241409 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a036218e-d3a4-4896-84d4-c195c6b587a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:40.363317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:40.400180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036218e-d3a4-4896-84d4-c195c6b587a8 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:40.521768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388efdb-7031-4231-b935-3c339b5b02c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:40.522339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388efdb-7031-4231-b935-3c339b5b02c8 service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:40.522617 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388efdb-7031-4231-b935-3c339b5b02c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:40.522886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388efdb-7031-4231-b935-3c339b5b02c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:40.524483 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f388efdb-7031-4231-b935-3c339b5b02c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:40.605197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:40.619501 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388efdb-7031-4231-b935-3c339b5b02c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:40.621101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388efdb-7031-4231-b935-3c339b5b02c8 service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:40.638716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 708/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:41.761354 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640fdab-bfcd-4841-911b-594cc374a607 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:41.761914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640fdab-bfcd-4841-911b-594cc374a607 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:41.761914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2640fdab-bfcd-4841-911b-594cc374a607 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:41.763154 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 16:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 16:01:41.770045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73ace7-130d-48e3-a2e7-ac7d7b47427d None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:41.770562 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73ace7-130d-48e3-a2e7-ac7d7b47427d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:41.770562 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73ace7-130d-48e3-a2e7-ac7d7b47427d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:41.867808 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-fc73ace7-130d-48e3-a2e7-ac7d7b47427d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:41.869035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-fc73ace7-130d-48e3-a2e7-ac7d7b47427d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:41.974938 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 16:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:42.133479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640fdab-bfcd-4841-911b-594cc374a607 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:42.135046 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2640fdab-bfcd-4841-911b-594cc374a607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:42.234287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:42.296845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2640fdab-bfcd-4841-911b-594cc374a607 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:42.427078 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db84fe-8d96-4044-8742-e9f378d76890 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:42.427625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db84fe-8d96-4044-8742-e9f378d76890 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:42.427898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db84fe-8d96-4044-8742-e9f378d76890 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:42.428162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db84fe-8d96-4044-8742-e9f378d76890 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:42.429811 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-64db84fe-8d96-4044-8742-e9f378d76890 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:42.492230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:42.543230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db84fe-8d96-4044-8742-e9f378d76890 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:42.544902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64db84fe-8d96-4044-8742-e9f378d76890 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:42.563228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 719/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:43.624486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a740f-ff6f-43f7-b3f3-91722d4a097a None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:43.625089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a740f-ff6f-43f7-b3f3-91722d4a097a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:43.625089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a740f-ff6f-43f7-b3f3-91722d4a097a None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:43.626447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 16:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:43.632994 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b74a5-a177-4974-8bd5-481dcfffc849 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:43.633477 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b74a5-a177-4974-8bd5-481dcfffc849 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:43.633477 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b74a5-a177-4974-8bd5-481dcfffc849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:43.712969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-d78b74a5-a177-4974-8bd5-481dcfffc849 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:43.714181 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d78b74a5-a177-4974-8bd5-481dcfffc849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:43.814905 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 16:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:43.965015 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a740f-ff6f-43f7-b3f3-91722d4a097a None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:43.967423 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-736a740f-ff6f-43f7-b3f3-91722d4a097a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:44.028613 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:44.137500 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a740f-ff6f-43f7-b3f3-91722d4a097a None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:44.257011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3f6db-28bb-48e5-847e-469334a415e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:44.257552 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3f6db-28bb-48e5-847e-469334a415e1 service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:44.257827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3f6db-28bb-48e5-847e-469334a415e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:44.258092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3f6db-28bb-48e5-847e-469334a415e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:44.259760 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-87e3f6db-28bb-48e5-847e-469334a415e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:44.267239 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:44.356020 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3f6db-28bb-48e5-847e-469334a415e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:44.357655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3f6db-28bb-48e5-847e-469334a415e1 service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:44.376812 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 711/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:45.427109 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1994c-d9bb-4f7a-865e-a7495de5f08f None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:45.427745 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1994c-d9bb-4f7a-865e-a7495de5f08f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:45.427745 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1994c-d9bb-4f7a-865e-a7495de5f08f None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:45.429034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 16:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:45.436360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46731e9-7206-472e-a30d-8ffa8e7261b4 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:45.437092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46731e9-7206-472e-a30d-8ffa8e7261b4 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:45.437092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46731e9-7206-472e-a30d-8ffa8e7261b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:45.524077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-c46731e9-7206-472e-a30d-8ffa8e7261b4 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:45.525286 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c46731e9-7206-472e-a30d-8ffa8e7261b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:45.607953 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:45.638536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 16:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:45.796242 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1994c-d9bb-4f7a-865e-a7495de5f08f None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:45.798011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-35a1994c-d9bb-4f7a-865e-a7495de5f08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:45.817079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:45.963242 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1994c-d9bb-4f7a-865e-a7495de5f08f None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:46.058857 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:46.084824 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34283058-b40d-455e-8594-0121395c5e2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:46.085429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34283058-b40d-455e-8594-0121395c5e2a service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:46.085695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34283058-b40d-455e-8594-0121395c5e2a service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:46.085959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34283058-b40d-455e-8594-0121395c5e2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:46.087598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-34283058-b40d-455e-8594-0121395c5e2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:46.213012 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34283058-b40d-455e-8594-0121395c5e2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:46.214600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34283058-b40d-455e-8594-0121395c5e2a service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:46.232483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 722/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:47.283197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cff47-6284-409f-91f4-a1e9522e2fae None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:47.284090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cff47-6284-409f-91f4-a1e9522e2fae None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:47.284090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cff47-6284-409f-91f4-a1e9522e2fae None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:47.286160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 16:01:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:47.292380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdcab9-d11b-4c02-9b2f-1b3ed577fb4a None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:47.292855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdcab9-d11b-4c02-9b2f-1b3ed577fb4a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:47.292960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fdcab9-d11b-4c02-9b2f-1b3ed577fb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:47.373134 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f0fdcab9-d11b-4c02-9b2f-1b3ed577fb4a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:47.374436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f0fdcab9-d11b-4c02-9b2f-1b3ed577fb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:47.420853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:47.480484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 16:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:47.629240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cff47-6284-409f-91f4-a1e9522e2fae None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:47.630803 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-908cff47-6284-409f-91f4-a1e9522e2fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:47.785420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cff47-6284-409f-91f4-a1e9522e2fae None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:47.843729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:47.910408 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe18cf2-533c-4a7e-a7b3-547e06fbd048 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:47.911028 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe18cf2-533c-4a7e-a7b3-547e06fbd048 service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:47.911303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe18cf2-533c-4a7e-a7b3-547e06fbd048 service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:47.911616 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe18cf2-533c-4a7e-a7b3-547e06fbd048 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:47.913275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5fe18cf2-533c-4a7e-a7b3-547e06fbd048 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:48.006537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe18cf2-533c-4a7e-a7b3-547e06fbd048 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:48.008172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe18cf2-533c-4a7e-a7b3-547e06fbd048 service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:48.026401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 714/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:49.086896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a246d84-e76b-4b50-9f0d-0b3c6ad955bd None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:49.087510 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a246d84-e76b-4b50-9f0d-0b3c6ad955bd None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:49.087510 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a246d84-e76b-4b50-9f0d-0b3c6ad955bd None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:49.088829 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 16:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:49.096290 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba045f8-6a98-43cd-9d4e-fb78dbe15867 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:49.097018 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba045f8-6a98-43cd-9d4e-fb78dbe15867 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:49.097178 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba045f8-6a98-43cd-9d4e-fb78dbe15867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:49.185355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-dba045f8-6a98-43cd-9d4e-fb78dbe15867 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:49.186775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-dba045f8-6a98-43cd-9d4e-fb78dbe15867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:49.188900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:49.419401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 16:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:49.570534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a246d84-e76b-4b50-9f0d-0b3c6ad955bd None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:49.573071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2a246d84-e76b-4b50-9f0d-0b3c6ad955bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:49.745839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a246d84-e76b-4b50-9f0d-0b3c6ad955bd None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:49.777410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:49.881879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bb611-a020-4ee0-ab19-75f6f53aca8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:49.882501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bb611-a020-4ee0-ab19-75f6f53aca8e service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:49.882778 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bb611-a020-4ee0-ab19-75f6f53aca8e service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:49.883048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bb611-a020-4ee0-ab19-75f6f53aca8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:49.884715 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-764bb611-a020-4ee0-ab19-75f6f53aca8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:49.978372 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764bb611-a020-4ee0-ab19-75f6f53aca8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:49.980001 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764bb611-a020-4ee0-ab19-75f6f53aca8e service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:49.999156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 725/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:52.050793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002bdee-d131-4a11-ae60-ffb05624e97b None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:52.051922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002bdee-d131-4a11-ae60-ffb05624e97b None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:52.051922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002bdee-d131-4a11-ae60-ffb05624e97b None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:52.053886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 16:01:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:52.060481 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ecfb4-541b-476b-b20c-8b877656a44d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:52.061023 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ecfb4-541b-476b-b20c-8b877656a44d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:52.061173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ecfb4-541b-476b-b20c-8b877656a44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:52.102780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:52.129086 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-135ecfb4-541b-476b-b20c-8b877656a44d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:52.130299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-135ecfb4-541b-476b-b20c-8b877656a44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:52.234689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 16:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:52.514720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002bdee-d131-4a11-ae60-ffb05624e97b None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:52.516304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b002bdee-d131-4a11-ae60-ffb05624e97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:52.627727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002bdee-d131-4a11-ae60-ffb05624e97b None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:52.660596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:52.710956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c32c0-b522-495b-84ce-1080245af485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:52.711589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c32c0-b522-495b-84ce-1080245af485 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:52.711858 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c32c0-b522-495b-84ce-1080245af485 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:52.712123 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c32c0-b522-495b-84ce-1080245af485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:52.713703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-7c3c32c0-b522-495b-84ce-1080245af485 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:52.828089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c32c0-b522-495b-84ce-1080245af485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:52.829486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c32c0-b522-495b-84ce-1080245af485 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:52.848044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 717/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:55.105492 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd3abb-a04f-4386-a4d2-71db04fdbc56 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:55.106094 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd3abb-a04f-4386-a4d2-71db04fdbc56 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:55.106094 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd3abb-a04f-4386-a4d2-71db04fdbc56 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:55.107372 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 16:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:55.113680 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc833f-e580-4c6c-8208-fbdaa5c95c47 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:55.114140 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc833f-e580-4c6c-8208-fbdaa5c95c47 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:55.114243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc833f-e580-4c6c-8208-fbdaa5c95c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:55.188646 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-bcdc833f-e580-4c6c-8208-fbdaa5c95c47 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:55.189861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-bcdc833f-e580-4c6c-8208-fbdaa5c95c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:55.203738 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:55.290035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 16:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:55.573989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd3abb-a04f-4386-a4d2-71db04fdbc56 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:55.575535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-cadd3abb-a04f-4386-a4d2-71db04fdbc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:55.683653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd3abb-a04f-4386-a4d2-71db04fdbc56 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:55.697457 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:55.770603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad300b5-08b8-4553-81b7-916e06aa89fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:55.771210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad300b5-08b8-4553-81b7-916e06aa89fd service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:55.771508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad300b5-08b8-4553-81b7-916e06aa89fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:55.771778 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad300b5-08b8-4553-81b7-916e06aa89fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:55.773371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-dad300b5-08b8-4553-81b7-916e06aa89fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:55.885208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad300b5-08b8-4553-81b7-916e06aa89fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:55.886579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad300b5-08b8-4553-81b7-916e06aa89fd service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:55.906020 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 728/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:01:58.401396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a282eb-86d3-40a9-9172-22f83ad3d3a4 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:58.401997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a282eb-86d3-40a9-9172-22f83ad3d3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:58.401997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a282eb-86d3-40a9-9172-22f83ad3d3a4 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:58.403289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 16:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:01:58.409685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16108b69-f798-48b5-a2b3-be936dc870e1 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:58.410159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16108b69-f798-48b5-a2b3-be936dc870e1 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:58.410271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16108b69-f798-48b5-a2b3-be936dc870e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:58.466969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:58.488454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-16108b69-f798-48b5-a2b3-be936dc870e1 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:01:58.489647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-16108b69-f798-48b5-a2b3-be936dc870e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:58.603906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 16:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:01:58.902530 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a282eb-86d3-40a9-9172-22f83ad3d3a4 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:01:58.905000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-43a282eb-86d3-40a9-9172-22f83ad3d3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:59.018750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a282eb-86d3-40a9-9172-22f83ad3d3a4 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:01:59.060036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:01:59.096628 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda74878-977e-44d9-acda-97c606b65524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:01:59.097237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda74878-977e-44d9-acda-97c606b65524 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:01:59.097508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda74878-977e-44d9-acda-97c606b65524 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:01:59.097773 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda74878-977e-44d9-acda-97c606b65524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:01:59.099363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-cda74878-977e-44d9-acda-97c606b65524 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:01:59.219297 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda74878-977e-44d9-acda-97c606b65524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:01:59.220768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda74878-977e-44d9-acda-97c606b65524 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:01:59.238307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 720/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:02:01.509107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200bf49-8c01-478a-bb9a-9286dcf31d6c None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:01.509760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200bf49-8c01-478a-bb9a-9286dcf31d6c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:01.509760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200bf49-8c01-478a-bb9a-9286dcf31d6c None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:01.511387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 16:02:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:02:01.518015 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4c318-0227-45c6-8a94-887c5d5776bf None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:01.518446 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4c318-0227-45c6-8a94-887c5d5776bf None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:01.518575 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4c318-0227-45c6-8a94-887c5d5776bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:01.590810 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-daa4c318-0227-45c6-8a94-887c5d5776bf None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:02:01.592099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-daa4c318-0227-45c6-8a94-887c5d5776bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:01.615010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:01.695839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 16:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:02:01.982307 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200bf49-8c01-478a-bb9a-9286dcf31d6c None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:02:01.984106 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8200bf49-8c01-478a-bb9a-9286dcf31d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:02.108954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200bf49-8c01-478a-bb9a-9286dcf31d6c None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:02:02.130621 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:02.188894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499b419-2a72-419a-8d66-8f4b65a97fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:02:02.189512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499b419-2a72-419a-8d66-8f4b65a97fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:02.189783 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499b419-2a72-419a-8d66-8f4b65a97fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:02.190050 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499b419-2a72-419a-8d66-8f4b65a97fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:02.191677 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5499b419-2a72-419a-8d66-8f4b65a97fc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:02.304709 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499b419-2a72-419a-8d66-8f4b65a97fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:02:02.306156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499b419-2a72-419a-8d66-8f4b65a97fc4 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:02:02.325533 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 731/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:02:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:02:04.578168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63993fe-ddef-456b-ae5c-26838fa83d2b None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:04.578772 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63993fe-ddef-456b-ae5c-26838fa83d2b None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:04.578772 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63993fe-ddef-456b-ae5c-26838fa83d2b None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:04.580219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 16:02:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:02:04.586637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165732ef-75b7-4aee-b55c-02c0b60072d9 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:04.587114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165732ef-75b7-4aee-b55c-02c0b60072d9 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:04.587212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165732ef-75b7-4aee-b55c-02c0b60072d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:04.654898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:04.659696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-165732ef-75b7-4aee-b55c-02c0b60072d9 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:02:04.662700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-165732ef-75b7-4aee-b55c-02c0b60072d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:04.766345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 16:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:02:05.055939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63993fe-ddef-456b-ae5c-26838fa83d2b None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:02:05.058371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f63993fe-ddef-456b-ae5c-26838fa83d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:05.176713 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63993fe-ddef-456b-ae5c-26838fa83d2b None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:02:05.231735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:05.260152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1296cd8-e567-4bf9-b8fc-f5f1f74d54fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:02:05.260763 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1296cd8-e567-4bf9-b8fc-f5f1f74d54fe service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:05.261037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1296cd8-e567-4bf9-b8fc-f5f1f74d54fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:05.261330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1296cd8-e567-4bf9-b8fc-f5f1f74d54fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:05.262911 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a1296cd8-e567-4bf9-b8fc-f5f1f74d54fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:05.376466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1296cd8-e567-4bf9-b8fc-f5f1f74d54fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:02:05.377823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1296cd8-e567-4bf9-b8fc-f5f1f74d54fe service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:02:05.396212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 723/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:02:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:02:07.603998 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17c9ec-0cc1-4fdb-ae99-edc7fed5ef94 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:07.604864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17c9ec-0cc1-4fdb-ae99-edc7fed5ef94 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:07.604864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17c9ec-0cc1-4fdb-ae99-edc7fed5ef94 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:07.606999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 16:02:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:02:07.613700 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf1722-30cb-4042-9ee2-731a75d8ebf4 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:07.614459 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf1722-30cb-4042-9ee2-731a75d8ebf4 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:07.614629 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf1722-30cb-4042-9ee2-731a75d8ebf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:07.686357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-31bf1722-30cb-4042-9ee2-731a75d8ebf4 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:02:07.687579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-31bf1722-30cb-4042-9ee2-731a75d8ebf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:07.719684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:07.798686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 16:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:02:08.081082 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17c9ec-0cc1-4fdb-ae99-edc7fed5ef94 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:02:08.083599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ff17c9ec-0cc1-4fdb-ae99-edc7fed5ef94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:08.201968 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17c9ec-0cc1-4fdb-ae99-edc7fed5ef94 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:02:08.232252 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:08.278492 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32ff51-a6ad-4d48-b0d2-915b924c19f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:02:08.279099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32ff51-a6ad-4d48-b0d2-915b924c19f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:08.279514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32ff51-a6ad-4d48-b0d2-915b924c19f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:08.279786 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32ff51-a6ad-4d48-b0d2-915b924c19f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:08.281423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0f32ff51-a6ad-4d48-b0d2-915b924c19f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:08.404577 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32ff51-a6ad-4d48-b0d2-915b924c19f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:02:08.405944 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32ff51-a6ad-4d48-b0d2-915b924c19f8 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:02:08.424977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 734/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:02:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:02:10.629096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374f5dd-6506-4b8d-93b5-0551773b54fc None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:10.629706 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374f5dd-6506-4b8d-93b5-0551773b54fc None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:10.629706 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374f5dd-6506-4b8d-93b5-0551773b54fc None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:10.631013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 16:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:02:10.637539 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c908732-2265-49c9-bbfc-12dfdb786d96 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:10.637973 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c908732-2265-49c9-bbfc-12dfdb786d96 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:10.638094 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c908732-2265-49c9-bbfc-12dfdb786d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:10.708187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-7c908732-2265-49c9-bbfc-12dfdb786d96 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:02:10.709416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-7c908732-2265-49c9-bbfc-12dfdb786d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:10.718137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:10.810247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 16:02:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:02:11.107793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374f5dd-6506-4b8d-93b5-0551773b54fc None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:02:11.109345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d374f5dd-6506-4b8d-93b5-0551773b54fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:11.221642 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374f5dd-6506-4b8d-93b5-0551773b54fc None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:02:11.293619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:11.318095 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22129f-9064-45aa-82ec-f10fbddb57e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:02:11.318720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22129f-9064-45aa-82ec-f10fbddb57e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:11.318999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22129f-9064-45aa-82ec-f10fbddb57e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:11.319274 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22129f-9064-45aa-82ec-f10fbddb57e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:11.320991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0a22129f-9064-45aa-82ec-f10fbddb57e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:11.442103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22129f-9064-45aa-82ec-f10fbddb57e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:02:11.443479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22129f-9064-45aa-82ec-f10fbddb57e5 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:02:11.462771 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 726/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:02:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:02:13.685460 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd1792-5bf2-4768-9081-b2d2afe5d0de None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:13.686064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd1792-5bf2-4768-9081-b2d2afe5d0de None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:13.686064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd1792-5bf2-4768-9081-b2d2afe5d0de None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:13.687427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 16:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:02:13.693992 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d109e26-e3b0-41b5-af5b-02c31677250e None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:13.694722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d109e26-e3b0-41b5-af5b-02c31677250e None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:13.694873 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d109e26-e3b0-41b5-af5b-02c31677250e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:13.772594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-2d109e26-e3b0-41b5-af5b-02c31677250e None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:02:13.773798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-2d109e26-e3b0-41b5-af5b-02c31677250e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:13.814936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:13.875082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 16:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:02:14.167504 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd1792-5bf2-4768-9081-b2d2afe5d0de None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:02:14.169934 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8fcd1792-5bf2-4768-9081-b2d2afe5d0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:14.280101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd1792-5bf2-4768-9081-b2d2afe5d0de None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:02:14.320045 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:14.365096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16210e78-e736-4299-bfa5-f6057da9b266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:02:14.365691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16210e78-e736-4299-bfa5-f6057da9b266 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:14.365966 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16210e78-e736-4299-bfa5-f6057da9b266 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:14.366238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16210e78-e736-4299-bfa5-f6057da9b266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:14.367925 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-16210e78-e736-4299-bfa5-f6057da9b266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:14.493079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16210e78-e736-4299-bfa5-f6057da9b266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:02:14.494461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16210e78-e736-4299-bfa5-f6057da9b266 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:02:14.513269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 737/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:02:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:02:16.650616 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068eda9c-c1f4-4151-964d-e6b53d59aa39 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:16.651220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068eda9c-c1f4-4151-964d-e6b53d59aa39 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:16.651220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068eda9c-c1f4-4151-964d-e6b53d59aa39 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:16.652615 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 16:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:02:16.658210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb4285-a62c-4575-9c6a-03da5a123dfd None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:16.658696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb4285-a62c-4575-9c6a-03da5a123dfd None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:16.658696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb4285-a62c-4575-9c6a-03da5a123dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:16.725874 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-ceeb4285-a62c-4575-9c6a-03da5a123dfd None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:02:16.727068 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ceeb4285-a62c-4575-9c6a-03da5a123dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:16.745384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:16.829532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 16:02:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:02:17.113035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068eda9c-c1f4-4151-964d-e6b53d59aa39 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:02:17.114671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-068eda9c-c1f4-4151-964d-e6b53d59aa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:17.224830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068eda9c-c1f4-4151-964d-e6b53d59aa39 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:02:17.297232 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:17.304611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783652b7-0e5a-47e8-99da-1dbe8dd5f952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:02:17.305225 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783652b7-0e5a-47e8-99da-1dbe8dd5f952 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:17.305491 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783652b7-0e5a-47e8-99da-1dbe8dd5f952 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:17.305762 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783652b7-0e5a-47e8-99da-1dbe8dd5f952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:17.307342 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-783652b7-0e5a-47e8-99da-1dbe8dd5f952 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:17.421544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783652b7-0e5a-47e8-99da-1dbe8dd5f952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:02:17.422879 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783652b7-0e5a-47e8-99da-1dbe8dd5f952 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:02:17.441127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 729/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:02:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:02:19.579799 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4312e-633c-44ea-8af4-629c3326a8b3 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:19.580381 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4312e-633c-44ea-8af4-629c3326a8b3 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:19.580381 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4312e-633c-44ea-8af4-629c3326a8b3 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:19.581699 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 16:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:02:19.588265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663850b9-8d10-45c6-8856-847f8582554d None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:19.588723 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663850b9-8d10-45c6-8856-847f8582554d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:19.588854 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663850b9-8d10-45c6-8856-847f8582554d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:19.660846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-663850b9-8d10-45c6-8856-847f8582554d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:02:19.663726 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-663850b9-8d10-45c6-8856-847f8582554d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:19.712973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:19.765376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 16:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:02:20.048305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4312e-633c-44ea-8af4-629c3326a8b3 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:02:20.050697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2cb4312e-633c-44ea-8af4-629c3326a8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:20.168272 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4312e-633c-44ea-8af4-629c3326a8b3 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:02:20.222022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:20.251059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843987a-3092-4f7d-a49c-28e142eeae4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:02:20.251704 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843987a-3092-4f7d-a49c-28e142eeae4f service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:20.251982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843987a-3092-4f7d-a49c-28e142eeae4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:20.252253 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843987a-3092-4f7d-a49c-28e142eeae4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:20.253863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5843987a-3092-4f7d-a49c-28e142eeae4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:20.377639 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843987a-3092-4f7d-a49c-28e142eeae4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:02:20.379091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843987a-3092-4f7d-a49c-28e142eeae4f service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:02:20.397860 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 740/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:02:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:02:29.763865 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=22 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:30.395255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=66 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:40.076549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba9666-45f5-42d0-92d9-0c9b351053a6 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:40.077861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba9666-45f5-42d0-92d9-0c9b351053a6 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:40.077861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba9666-45f5-42d0-92d9-0c9b351053a6 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:40.080053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 731/1471] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:02:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:02:40.089194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717ab61-9fcc-4ac3-b96a-f34d91605d6d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:40.089928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717ab61-9fcc-4ac3-b96a-f34d91605d6d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:40.090092 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2717ab61-9fcc-4ac3-b96a-f34d91605d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:40.163185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-2717ab61-9fcc-4ac3-b96a-f34d91605d6d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:02:40.164489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2717ab61-9fcc-4ac3-b96a-f34d91605d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:40.267386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 741/1472] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:02:40.569547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235439e-fa57-43ae-8247-f6fa93dd83b8 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:40.569831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235439e-fa57-43ae-8247-f6fa93dd83b8 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:40.570101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d235439e-fa57-43ae-8247-f6fa93dd83b8 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:40.571462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 732/1473] 10.222.0.27 () {54 vars in 966 bytes} [Thu Sep 8 16:02:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:02:40.576995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16576f0-6c8b-4c07-bb8c-d700ad901543 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:40.577282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16576f0-6c8b-4c07-bb8c-d700ad901543 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:40.577534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16576f0-6c8b-4c07-bb8c-d700ad901543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:40.663411 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-e16576f0-6c8b-4c07-bb8c-d700ad901543 None None] MFA Rules not processed for user `f24ee16c043244938901c4e850dea967`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:02:40.664688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e16576f0-6c8b-4c07-bb8c-d700ad901543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:40.687765 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:40.747685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 742/1474] 10.222.0.27 () {58 vars in 1063 bytes} [Thu Sep 8 16:02:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:02:40.755879 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235439e-fa57-43ae-8247-f6fa93dd83b8 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:02:40.757526 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d235439e-fa57-43ae-8247-f6fa93dd83b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:40.870521 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d235439e-fa57-43ae-8247-f6fa93dd83b8 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:02:40.955683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01552f23-89e1-441f-a3ea-2df72f9ac9bc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f24ee16c043244938901c4e850dea967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:02:40.956295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01552f23-89e1-441f-a3ea-2df72f9ac9bc service glance] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:40.956565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01552f23-89e1-441f-a3ea-2df72f9ac9bc service glance] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:40.956834 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01552f23-89e1-441f-a3ea-2df72f9ac9bc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:40.958417 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-01552f23-89e1-441f-a3ea-2df72f9ac9bc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:41.020361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:41.088923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01552f23-89e1-441f-a3ea-2df72f9ac9bc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:02:41.090365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01552f23-89e1-441f-a3ea-2df72f9ac9bc service glance] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:02:41.109653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 733/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:02:43.354458 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ebae5-ab00-4c54-bd4e-93f01fdbe43a None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:43.355163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ebae5-ab00-4c54-bd4e-93f01fdbe43a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:43.355163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ebae5-ab00-4c54-bd4e-93f01fdbe43a None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:43.357184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 743/1476] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:02:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:02:43.363120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688e94c-338f-4781-9ff5-46f3e92b2bb7 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:43.363676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688e94c-338f-4781-9ff5-46f3e92b2bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:43.363676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688e94c-338f-4781-9ff5-46f3e92b2bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:43.431112 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-f688e94c-338f-4781-9ff5-46f3e92b2bb7 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:02:43.432361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f688e94c-338f-4781-9ff5-46f3e92b2bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:43.531597 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 734/1477] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:02:43.806167 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ebae5-ab00-4c54-bd4e-93f01fdbe43a None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:02:43.808523 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ed8ebae5-ab00-4c54-bd4e-93f01fdbe43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:43.940058 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ebae5-ab00-4c54-bd4e-93f01fdbe43a None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:02:43.992625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:44.036534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8789715-54d5-41f5-8467-1a8a2cc1835c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:02:44.037174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8789715-54d5-41f5-8467-1a8a2cc1835c admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:44.037448 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8789715-54d5-41f5-8467-1a8a2cc1835c admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:44.037714 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8789715-54d5-41f5-8467-1a8a2cc1835c admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:44.038389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8789715-54d5-41f5-8467-1a8a2cc1835c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:02:44.040257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: WARNING py.warnings [None req-e8789715-54d5-41f5-8467-1a8a2cc1835c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 16:02:44.040257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: warnings.warn(msg) Sep 08 16:02:44.040257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: Sep 08 16:02:44.041402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8789715-54d5-41f5-8467-1a8a2cc1835c admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:02:44.061391 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 744/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Thu Sep 8 16:02:43 2022] GET /identity/v3/projects => generated 2553 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:02:46.236362 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be056f-afa2-4f0e-a90b-46250acec8b0 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:46.236888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be056f-afa2-4f0e-a90b-46250acec8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:46.236888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be056f-afa2-4f0e-a90b-46250acec8b0 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:46.238280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 735/1479] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:02:46.244818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892afe8b-c383-4265-8360-efe1f43b419f None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:46.245283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892afe8b-c383-4265-8360-efe1f43b419f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:46.245387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892afe8b-c383-4265-8360-efe1f43b419f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:46.322791 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-892afe8b-c383-4265-8360-efe1f43b419f None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:02:46.324042 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-892afe8b-c383-4265-8360-efe1f43b419f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:46.433436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 745/1480] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:02:46.735100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be056f-afa2-4f0e-a90b-46250acec8b0 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:02:46.737143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-90be056f-afa2-4f0e-a90b-46250acec8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:46.749295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:46.895276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be056f-afa2-4f0e-a90b-46250acec8b0 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:02:46.983518 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:47.019349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e41c433-6f15-47b9-904c-f62b1ce1e102 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:02:47.019980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41c433-6f15-47b9-904c-f62b1ce1e102 service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:47.020251 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41c433-6f15-47b9-904c-f62b1ce1e102 service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:47.020510 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e41c433-6f15-47b9-904c-f62b1ce1e102 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:47.022103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5e41c433-6f15-47b9-904c-f62b1ce1e102 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:47.116210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41c433-6f15-47b9-904c-f62b1ce1e102 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:02:47.117800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e41c433-6f15-47b9-904c-f62b1ce1e102 service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:02:47.135868 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 736/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:02:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:02:49.459205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beefb87-a28c-4ace-9e42-95a3154a556a None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:49.460106 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beefb87-a28c-4ace-9e42-95a3154a556a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:49.460106 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beefb87-a28c-4ace-9e42-95a3154a556a None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:49.462178 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 746/1482] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:02:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:02:49.468654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d872b-ae11-4185-a323-529e55fc363e None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:49.469368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d872b-ae11-4185-a323-529e55fc363e None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:49.469509 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d872b-ae11-4185-a323-529e55fc363e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:49.542278 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-af5d872b-ae11-4185-a323-529e55fc363e None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:02:49.543474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-af5d872b-ae11-4185-a323-529e55fc363e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:49.592738 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:49.649079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 737/1483] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:02:49.941438 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beefb87-a28c-4ace-9e42-95a3154a556a None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:02:49.942999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2beefb87-a28c-4ace-9e42-95a3154a556a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:49.968851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:50.052238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beefb87-a28c-4ace-9e42-95a3154a556a None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:02:50.147055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eae8f1-09ad-44af-a26c-8a17021dbce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:02:50.147698 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eae8f1-09ad-44af-a26c-8a17021dbce8 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:50.147981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eae8f1-09ad-44af-a26c-8a17021dbce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:50.148255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eae8f1-09ad-44af-a26c-8a17021dbce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:50.149870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-24eae8f1-09ad-44af-a26c-8a17021dbce8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:50.231177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:50.270434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eae8f1-09ad-44af-a26c-8a17021dbce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:02:50.271968 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eae8f1-09ad-44af-a26c-8a17021dbce8 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:02:50.291224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 747/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:02:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:02:52.483278 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae583e-8d5b-4afb-ab2b-9c0e10aaeebf None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:52.483825 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae583e-8d5b-4afb-ab2b-9c0e10aaeebf None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:52.483825 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae583e-8d5b-4afb-ab2b-9c0e10aaeebf None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:52.485845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 738/1485] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:02:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:02:52.494261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6a1b0-bf0d-4321-84e2-83dad5d4d215 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:52.494744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6a1b0-bf0d-4321-84e2-83dad5d4d215 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:52.494744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6a1b0-bf0d-4321-84e2-83dad5d4d215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:52.563788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f2c6a1b0-bf0d-4321-84e2-83dad5d4d215 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:02:52.564998 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f2c6a1b0-bf0d-4321-84e2-83dad5d4d215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:52.670617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 748/1486] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:02:52.955334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae583e-8d5b-4afb-ab2b-9c0e10aaeebf None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:02:52.956949 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-64ae583e-8d5b-4afb-ab2b-9c0e10aaeebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:53.070879 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae583e-8d5b-4afb-ab2b-9c0e10aaeebf None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:02:53.162173 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:53.163991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927b63b-2c0d-4052-a344-c53535981c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:02:53.165235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927b63b-2c0d-4052-a344-c53535981c65 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:53.165546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927b63b-2c0d-4052-a344-c53535981c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:53.169336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927b63b-2c0d-4052-a344-c53535981c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:53.169336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4927b63b-2c0d-4052-a344-c53535981c65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:53.297644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927b63b-2c0d-4052-a344-c53535981c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:02:53.299105 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927b63b-2c0d-4052-a344-c53535981c65 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:02:53.318873 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 739/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:02:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:02:55.688456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e4fec-970b-4edd-9fc9-1e0fd88cdb41 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:55.689171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e4fec-970b-4edd-9fc9-1e0fd88cdb41 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:55.689171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21e4fec-970b-4edd-9fc9-1e0fd88cdb41 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:55.690409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 749/1488] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:02:55.696942 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da2ddc-db39-4e27-a436-935a237502e8 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:55.697726 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da2ddc-db39-4e27-a436-935a237502e8 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:55.697881 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da2ddc-db39-4e27-a436-935a237502e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:55.776254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-27da2ddc-db39-4e27-a436-935a237502e8 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:02:55.777456 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-27da2ddc-db39-4e27-a436-935a237502e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:55.833894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:55.881032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 740/1489] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:02:56.168587 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e4fec-970b-4edd-9fc9-1e0fd88cdb41 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:02:56.170154 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a21e4fec-970b-4edd-9fc9-1e0fd88cdb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:56.220760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:56.288199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21e4fec-970b-4edd-9fc9-1e0fd88cdb41 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:02:56.363372 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66924747-152d-484d-b81d-55757e30675e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:02:56.364019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66924747-152d-484d-b81d-55757e30675e service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:56.364292 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66924747-152d-484d-b81d-55757e30675e service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:56.364560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66924747-152d-484d-b81d-55757e30675e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:56.366214 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-66924747-152d-484d-b81d-55757e30675e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:56.452443 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:56.482325 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66924747-152d-484d-b81d-55757e30675e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:02:56.483790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66924747-152d-484d-b81d-55757e30675e service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:02:56.626247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 750/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:02:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:02:58.984326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11529c-30d3-4f1e-b128-d74b0e7d2590 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:58.985269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11529c-30d3-4f1e-b128-d74b0e7d2590 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:58.985269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11529c-30d3-4f1e-b128-d74b0e7d2590 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:58.986797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 741/1491] 10.222.0.27 () {54 vars in 927 bytes} [Thu Sep 8 16:02:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:02:58.992209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33397855-17fb-4b5b-b0e2-5953a5b2f74d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:58.992665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33397855-17fb-4b5b-b0e2-5953a5b2f74d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:58.992763 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33397855-17fb-4b5b-b0e2-5953a5b2f74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:59.081933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-33397855-17fb-4b5b-b0e2-5953a5b2f74d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:02:59.083168 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-33397855-17fb-4b5b-b0e2-5953a5b2f74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:59.179857 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 751/1492] 10.222.0.27 () {58 vars in 1024 bytes} [Thu Sep 8 16:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:02:59.473112 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11529c-30d3-4f1e-b128-d74b0e7d2590 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:02:59.474615 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6c11529c-30d3-4f1e-b128-d74b0e7d2590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:59.584230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11529c-30d3-4f1e-b128-d74b0e7d2590 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:02:59.665117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efaf2d7-c764-4956-be73-692e58e81f67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:02:59.665931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efaf2d7-c764-4956-be73-692e58e81f67 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:02:59.666242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efaf2d7-c764-4956-be73-692e58e81f67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:02:59.666639 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efaf2d7-c764-4956-be73-692e58e81f67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:02:59.669073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0efaf2d7-c764-4956-be73-692e58e81f67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:02:59.679656 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:02:59.792946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efaf2d7-c764-4956-be73-692e58e81f67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:02:59.794393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efaf2d7-c764-4956-be73-692e58e81f67 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:02:59.814973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 742/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:02:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:03:09.178200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=82 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:03:09.812536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=50 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:04:24.447112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a60839-4056-4795-92cc-2d2f8fb1ef34 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:24.447695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a60839-4056-4795-92cc-2d2f8fb1ef34 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:24.447695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a60839-4056-4795-92cc-2d2f8fb1ef34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:24.529477 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-99a60839-4056-4795-92cc-2d2f8fb1ef34 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:04:24.530661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-99a60839-4056-4795-92cc-2d2f8fb1ef34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:24.637947 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 752/1494] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:04:24.641292 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff308a57-5cf6-4de4-bb49-69c10ed8d187 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:24.641750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff308a57-5cf6-4de4-bb49-69c10ed8d187 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:24.641853 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff308a57-5cf6-4de4-bb49-69c10ed8d187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:24.718781 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-ff308a57-5cf6-4de4-bb49-69c10ed8d187 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:04:24.719937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ff308a57-5cf6-4de4-bb49-69c10ed8d187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:24.820676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 743/1495] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:04:24.826056 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a60839-4056-4795-92cc-2d2f8fb1ef34 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:04:24.827273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-99a60839-4056-4795-92cc-2d2f8fb1ef34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:24.839270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a60839-4056-4795-92cc-2d2f8fb1ef34 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:04:24.904500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:04:24.937932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e899e7-d3f6-493b-9dbb-3a0452daf328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:04:24.938538 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e899e7-d3f6-493b-9dbb-3a0452daf328 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:24.938870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e899e7-d3f6-493b-9dbb-3a0452daf328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:24.939135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e899e7-d3f6-493b-9dbb-3a0452daf328 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:24.939812 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e899e7-d3f6-493b-9dbb-3a0452daf328 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:04:24.940101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e899e7-d3f6-493b-9dbb-3a0452daf328 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:04:24.941123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: WARNING py.warnings [None req-e0e899e7-d3f6-493b-9dbb-3a0452daf328 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 16:04:24.941123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: warnings.warn(msg) Sep 08 16:04:24.941123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: Sep 08 16:04:24.941999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e899e7-d3f6-493b-9dbb-3a0452daf328 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:04:24.954387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 753/1496] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:04:24.963067 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff308a57-5cf6-4de4-bb49-69c10ed8d187 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:04:24.972470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff308a57-5cf6-4de4-bb49-69c10ed8d187 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:04:25.035922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:04:25.072432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633df2e-eaf3-4435-bd45-b3d6cf4b6080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:04:25.073046 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633df2e-eaf3-4435-bd45-b3d6cf4b6080 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:25.073319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633df2e-eaf3-4435-bd45-b3d6cf4b6080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:25.073587 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633df2e-eaf3-4435-bd45-b3d6cf4b6080 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:25.074232 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633df2e-eaf3-4435-bd45-b3d6cf4b6080 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:04:25.076359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: WARNING py.warnings [None req-6633df2e-eaf3-4435-bd45-b3d6cf4b6080 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 16:04:25.076359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: warnings.warn(msg) Sep 08 16:04:25.076359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: Sep 08 16:04:25.077556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633df2e-eaf3-4435-bd45-b3d6cf4b6080 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:04:25.137652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 744/1497] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:04:24 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:04:25.142165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e899e7-d3f6-493b-9dbb-3a0452daf328 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:04:25.143785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e0e899e7-d3f6-493b-9dbb-3a0452daf328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:25.178734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e899e7-d3f6-493b-9dbb-3a0452daf328 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:04:25.279679 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847aa1e2-a24f-4509-ac8f-89f106bf331f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:04:25.280302 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aa1e2-a24f-4509-ac8f-89f106bf331f admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:25.280581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aa1e2-a24f-4509-ac8f-89f106bf331f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:25.280863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847aa1e2-a24f-4509-ac8f-89f106bf331f admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:25.281488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847aa1e2-a24f-4509-ac8f-89f106bf331f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:04:25.282514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: WARNING py.warnings [None req-847aa1e2-a24f-4509-ac8f-89f106bf331f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 16:04:25.282514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: warnings.warn(msg) Sep 08 16:04:25.282514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: Sep 08 16:04:25.283785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847aa1e2-a24f-4509-ac8f-89f106bf331f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:04:25.331124 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 754/1498] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:04:25 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:04:25.335505 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633df2e-eaf3-4435-bd45-b3d6cf4b6080 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:04:25.337084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6633df2e-eaf3-4435-bd45-b3d6cf4b6080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:25.404285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633df2e-eaf3-4435-bd45-b3d6cf4b6080 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:04:25.490595 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:04:25.502464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02388b2b-eb97-4a3f-814a-b8d3d2b70ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:04:25.503068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02388b2b-eb97-4a3f-814a-b8d3d2b70ece admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:25.503345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02388b2b-eb97-4a3f-814a-b8d3d2b70ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:25.503650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02388b2b-eb97-4a3f-814a-b8d3d2b70ece admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:25.504301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02388b2b-eb97-4a3f-814a-b8d3d2b70ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:04:25.505755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: WARNING py.warnings [None req-02388b2b-eb97-4a3f-814a-b8d3d2b70ece admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 16:04:25.505755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: warnings.warn(msg) Sep 08 16:04:25.505755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: Sep 08 16:04:25.506697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02388b2b-eb97-4a3f-814a-b8d3d2b70ece admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:04:25.518964 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 745/1499] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:04:25.523203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847aa1e2-a24f-4509-ac8f-89f106bf331f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:04:25.524734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-847aa1e2-a24f-4509-ac8f-89f106bf331f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:25.536705 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847aa1e2-a24f-4509-ac8f-89f106bf331f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:04:25.563623 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:04:25.636498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a23ea66-4a9f-401b-b411-6ae346e26f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:04:25.637112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23ea66-4a9f-401b-b411-6ae346e26f68 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:25.637385 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23ea66-4a9f-401b-b411-6ae346e26f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:25.637678 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23ea66-4a9f-401b-b411-6ae346e26f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:25.638324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23ea66-4a9f-401b-b411-6ae346e26f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:04:25.639341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: WARNING py.warnings [None req-5a23ea66-4a9f-401b-b411-6ae346e26f68 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 16:04:25.639341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: warnings.warn(msg) Sep 08 16:04:25.639341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: Sep 08 16:04:25.640256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a23ea66-4a9f-401b-b411-6ae346e26f68 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:04:25.653119 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 755/1500] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:04:25.657815 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02388b2b-eb97-4a3f-814a-b8d3d2b70ece admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:04:25.659371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-02388b2b-eb97-4a3f-814a-b8d3d2b70ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:25.696794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02388b2b-eb97-4a3f-814a-b8d3d2b70ece admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:04:25.810432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba471dc-d265-41f6-bfd3-b5c87f0366dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:04:25.811024 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba471dc-d265-41f6-bfd3-b5c87f0366dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:25.811299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba471dc-d265-41f6-bfd3-b5c87f0366dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:25.811608 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba471dc-d265-41f6-bfd3-b5c87f0366dc admin admin] PATH_INFO: `/v3/projects/818c619856cf4773a7f5c73ea3b1a3d5/users/03f0fb81f1b94455a0cfe73a47cef109/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:25.836355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba471dc-d265-41f6-bfd3-b5c87f0366dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=818c619856cf4773a7f5c73ea3b1a3d5, user_id=03f0fb81f1b94455a0cfe73a47cef109, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:04:25.837382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: WARNING py.warnings [None req-8ba471dc-d265-41f6-bfd3-b5c87f0366dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 16:04:25.837382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: warnings.warn(msg) Sep 08 16:04:25.837382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: Sep 08 16:04:25.839339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba471dc-d265-41f6-bfd3-b5c87f0366dc admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:04:25.867377 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 746/1501] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:04:25 2022] PUT /identity/v3/projects/818c619856cf4773a7f5c73ea3b1a3d5/users/03f0fb81f1b94455a0cfe73a47cef109/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:04:25.879970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f21b15-307b-4e75-9176-78060efe1ffd None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:25.880249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f21b15-307b-4e75-9176-78060efe1ffd None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:25.880518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f21b15-307b-4e75-9176-78060efe1ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:25.937489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-c7f21b15-307b-4e75-9176-78060efe1ffd None None] MFA Rules not processed for user `03f0fb81f1b94455a0cfe73a47cef109`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:04:25.938698 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c7f21b15-307b-4e75-9176-78060efe1ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:26.021155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 756/1502] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:04:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:04:26.035043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91fe5b-736e-4cd1-912a-c54c9be3d510 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:26.035334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91fe5b-736e-4cd1-912a-c54c9be3d510 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:26.035647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91fe5b-736e-4cd1-912a-c54c9be3d510 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:26.037005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 747/1503] 10.222.0.27 () {54 vars in 964 bytes} [Thu Sep 8 16:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:04:26.041823 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5026b-6b3f-4e1f-b1f8-372d11057cd2 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:26.042098 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5026b-6b3f-4e1f-b1f8-372d11057cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:26.042365 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5026b-6b3f-4e1f-b1f8-372d11057cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:26.076155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:04:26.122436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-4cd5026b-6b3f-4e1f-b1f8-372d11057cd2 None None] MFA Rules not processed for user `89df988d73ff48f3b7a92d335df9b633`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:04:26.123654 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-4cd5026b-6b3f-4e1f-b1f8-372d11057cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:26.202909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 757/1504] 10.222.0.27 () {58 vars in 1061 bytes} [Thu Sep 8 16:04:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:04:26.211143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91fe5b-736e-4cd1-912a-c54c9be3d510 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:04:26.212831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-fa91fe5b-736e-4cd1-912a-c54c9be3d510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:26.264727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:04:26.324617 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91fe5b-736e-4cd1-912a-c54c9be3d510 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:04:26.406207 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b57fe-a044-4557-9e51-971573bbb38f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:04:26.406808 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b57fe-a044-4557-9e51-971573bbb38f service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:26.407079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b57fe-a044-4557-9e51-971573bbb38f service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:26.407354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572b57fe-a044-4557-9e51-971573bbb38f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:26.408975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-572b57fe-a044-4557-9e51-971573bbb38f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:26.519822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b57fe-a044-4557-9e51-971573bbb38f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:04:26.521258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572b57fe-a044-4557-9e51-971573bbb38f service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:04:26.524817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:04:26.540230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 748/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:04:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:04:26.588201 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d0578-4fdd-4218-b2ef-9c84014c967d None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:26.588652 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d0578-4fdd-4218-b2ef-9c84014c967d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:26.588764 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d0578-4fdd-4218-b2ef-9c84014c967d None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:26.589976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 758/1506] 10.222.0.27 () {52 vars in 830 bytes} [Thu Sep 8 16:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:04:26.593676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b57fe-a044-4557-9e51-971573bbb38f service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:04:26.594960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-572b57fe-a044-4557-9e51-971573bbb38f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:26.702201 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572b57fe-a044-4557-9e51-971573bbb38f service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:04:26.799710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57489c02-3b4a-44e4-9ac6-b1657f9006d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:04:26.800357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57489c02-3b4a-44e4-9ac6-b1657f9006d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:26.800616 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57489c02-3b4a-44e4-9ac6-b1657f9006d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:26.800902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57489c02-3b4a-44e4-9ac6-b1657f9006d8 admin admin] PATH_INFO: `/v3/users/03f0fb81f1b94455a0cfe73a47cef109` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:26.815127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57489c02-3b4a-44e4-9ac6-b1657f9006d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f0fb81f1b94455a0cfe73a47cef109)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:04:26.816202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: WARNING py.warnings [None req-57489c02-3b4a-44e4-9ac6-b1657f9006d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 16:04:26.816202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: warnings.warn(msg) Sep 08 16:04:26.816202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: Sep 08 16:04:26.817430 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57489c02-3b4a-44e4-9ac6-b1657f9006d8 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:04:26.828313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:04:26.890233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-57489c02-3b4a-44e4-9ac6-b1657f9006d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f0fb81f1b94455a0cfe73a47cef109'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:04:26.902819 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-57489c02-3b4a-44e4-9ac6-b1657f9006d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f0fb81f1b94455a0cfe73a47cef109'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:04:26.903275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-57489c02-3b4a-44e4-9ac6-b1657f9006d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f0fb81f1b94455a0cfe73a47cef109'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:04:26.903801 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-57489c02-3b4a-44e4-9ac6-b1657f9006d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f0fb81f1b94455a0cfe73a47cef109'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:04:26.918824 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-57489c02-3b4a-44e4-9ac6-b1657f9006d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f0fb81f1b94455a0cfe73a47cef109'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:04:26.948625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 749/1507] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:04:26 2022] DELETE /identity/v3/users/03f0fb81f1b94455a0cfe73a47cef109 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:04:26.953630 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6387d9e-54f8-44d9-8faf-2082fc9b9312 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:26.953849 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6387d9e-54f8-44d9-8faf-2082fc9b9312 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:26.954113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6387d9e-54f8-44d9-8faf-2082fc9b9312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:27.015097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-e6387d9e-54f8-44d9-8faf-2082fc9b9312 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:04:27.019252 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e6387d9e-54f8-44d9-8faf-2082fc9b9312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:27.115914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:04:27.129708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 759/1508] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:04:27.140634 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57489c02-3b4a-44e4-9ac6-b1657f9006d8 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:04:27.141881 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-57489c02-3b4a-44e4-9ac6-b1657f9006d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:27.251951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57489c02-3b4a-44e4-9ac6-b1657f9006d8 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:04:27.312041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:04:27.334243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c4219c-a405-48ce-9dab-05c7d573034a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:04:27.334864 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4219c-a405-48ce-9dab-05c7d573034a service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:27.335133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4219c-a405-48ce-9dab-05c7d573034a service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:27.335397 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c4219c-a405-48ce-9dab-05c7d573034a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:27.337098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c6c4219c-a405-48ce-9dab-05c7d573034a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:27.459030 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4219c-a405-48ce-9dab-05c7d573034a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:04:27.460568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c4219c-a405-48ce-9dab-05c7d573034a service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:04:27.479847 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 750/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:04:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:04:27.814570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6387d9e-54f8-44d9-8faf-2082fc9b9312 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:04:27.816667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e6387d9e-54f8-44d9-8faf-2082fc9b9312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:04:27.829288 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6387d9e-54f8-44d9-8faf-2082fc9b9312 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:04:27.927805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352532f0-0816-4df3-ba7f-af47ee03bd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:04:27.928483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352532f0-0816-4df3-ba7f-af47ee03bd94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:04:27.928754 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352532f0-0816-4df3-ba7f-af47ee03bd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:04:27.929019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352532f0-0816-4df3-ba7f-af47ee03bd94 admin admin] PATH_INFO: `/v3/projects/818c619856cf4773a7f5c73ea3b1a3d5` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:04:27.939070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352532f0-0816-4df3-ba7f-af47ee03bd94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=818c619856cf4773a7f5c73ea3b1a3d5)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:04:27.940176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: WARNING py.warnings [None req-352532f0-0816-4df3-ba7f-af47ee03bd94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 16:04:27.940176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: warnings.warn(msg) Sep 08 16:04:27.940176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: Sep 08 16:04:27.941395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352532f0-0816-4df3-ba7f-af47ee03bd94 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:04:28.005076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-352532f0-0816-4df3-ba7f-af47ee03bd94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '818c619856cf4773a7f5c73ea3b1a3d5', 'request_id': 'req-352532f0-0816-4df3-ba7f-af47ee03bd94'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:04:28.016794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-352532f0-0816-4df3-ba7f-af47ee03bd94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '818c619856cf4773a7f5c73ea3b1a3d5', 'request_id': 'req-352532f0-0816-4df3-ba7f-af47ee03bd94'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:04:28.024804 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-352532f0-0816-4df3-ba7f-af47ee03bd94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '818c619856cf4773a7f5c73ea3b1a3d5', 'request_id': 'req-352532f0-0816-4df3-ba7f-af47ee03bd94'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:04:28.054321 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-352532f0-0816-4df3-ba7f-af47ee03bd94 admin admin] The token cache is being invalidate because project 818c619856cf4773a7f5c73ea3b1a3d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:04:28.054594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-352532f0-0816-4df3-ba7f-af47ee03bd94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:04:28.055012 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-352532f0-0816-4df3-ba7f-af47ee03bd94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:04:28.056440 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 760/1510] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:04:27 2022] DELETE /identity/v3/projects/818c619856cf4773a7f5c73ea3b1a3d5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:04:37.477410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=62 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:04:38.052156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:05:52.465767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93961c8d-4147-416d-8ac3-1ab7e5f436ff None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:05:52.466316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93961c8d-4147-416d-8ac3-1ab7e5f436ff None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:05:52.466316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93961c8d-4147-416d-8ac3-1ab7e5f436ff None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:05:52.467637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 16:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:05:52.472861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b9af0-b14e-4ed8-938f-0ab2ebf9b2ca None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:05:52.473318 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b9af0-b14e-4ed8-938f-0ab2ebf9b2ca None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:05:52.473425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b9af0-b14e-4ed8-938f-0ab2ebf9b2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:05:52.560001 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-324b9af0-b14e-4ed8-938f-0ab2ebf9b2ca None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:05:52.561962 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-324b9af0-b14e-4ed8-938f-0ab2ebf9b2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:05:52.673906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 16:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:05:52.828018 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93961c8d-4147-416d-8ac3-1ab7e5f436ff None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:05:52.832089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-93961c8d-4147-416d-8ac3-1ab7e5f436ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:05:52.961084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93961c8d-4147-416d-8ac3-1ab7e5f436ff None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:05:52.967851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae0d00-8987-40be-96ef-df2915f614c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:05:52.968473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae0d00-8987-40be-96ef-df2915f614c7 service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:05:52.968754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae0d00-8987-40be-96ef-df2915f614c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:05:52.969027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae0d00-8987-40be-96ef-df2915f614c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:05:52.978408 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae0d00-8987-40be-96ef-df2915f614c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:05:52.980137 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae0d00-8987-40be-96ef-df2915f614c7 service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:05:52.990290 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 752/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:05:54.067772 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51529628-d2f5-4d13-b02a-5f3f088a6d39 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:05:54.068382 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51529628-d2f5-4d13-b02a-5f3f088a6d39 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:05:54.068382 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51529628-d2f5-4d13-b02a-5f3f088a6d39 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:05:54.069679 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 16:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:05:54.077609 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7396498-6667-4d7e-970b-da6741df2157 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:05:54.078078 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7396498-6667-4d7e-970b-da6741df2157 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:05:54.078412 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7396498-6667-4d7e-970b-da6741df2157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:05:54.099175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-f7396498-6667-4d7e-970b-da6741df2157 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:05:54.100463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f7396498-6667-4d7e-970b-da6741df2157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:05:54.109528 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 16:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:05:54.262963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51529628-d2f5-4d13-b02a-5f3f088a6d39 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:05:54.283241 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51529628-d2f5-4d13-b02a-5f3f088a6d39 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:05:54.290757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96222aa8-9311-4b44-b3d0-aa74ed85ab87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:05:54.291375 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96222aa8-9311-4b44-b3d0-aa74ed85ab87 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:05:54.291680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96222aa8-9311-4b44-b3d0-aa74ed85ab87 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:05:54.291952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96222aa8-9311-4b44-b3d0-aa74ed85ab87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:05:54.300714 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96222aa8-9311-4b44-b3d0-aa74ed85ab87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:05:54.302359 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96222aa8-9311-4b44-b3d0-aa74ed85ab87 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:05:54.312054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 763/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:05:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:05:54.637189 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2b25e-259b-48c8-9751-6b515a54074d None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:05:54.637416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2b25e-259b-48c8-9751-6b515a54074d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:05:54.637692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b2b25e-259b-48c8-9751-6b515a54074d None None] PATH_INFO: `/v3` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:05:54.639149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 754/1517] 10.222.0.20 () {54 vars in 925 bytes} [Thu Sep 8 16:05:54 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:05:54.645852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29a606-e062-4ec0-87cf-72d67a0cc31a None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:05:54.646162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29a606-e062-4ec0-87cf-72d67a0cc31a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:05:54.646385 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29a606-e062-4ec0-87cf-72d67a0cc31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:05:54.709449 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-8e29a606-e062-4ec0-87cf-72d67a0cc31a None None] MFA Rules not processed for user `dd28bb31b55e45668212c9ab69ed3ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:05:54.710675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8e29a606-e062-4ec0-87cf-72d67a0cc31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:05:54.728133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:05:54.785448 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 764/1518] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Sep 8 16:05:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:05:58.356575 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1bfc9-b775-4e7d-a7bd-84754dd9c307 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:05:58.357077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1bfc9-b775-4e7d-a7bd-84754dd9c307 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:05:58.357077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1bfc9-b775-4e7d-a7bd-84754dd9c307 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:05:58.358445 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 755/1519] 10.222.0.27 () {54 vars in 961 bytes} [Thu Sep 8 16:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:05:58.363402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a3f00-039f-4127-a27b-8ee3aec36230 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:05:58.363886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a3f00-039f-4127-a27b-8ee3aec36230 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:05:58.363993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a3f00-039f-4127-a27b-8ee3aec36230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:05:58.384100 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-761a3f00-039f-4127-a27b-8ee3aec36230 None None] MFA Rules not processed for user `dd28bb31b55e45668212c9ab69ed3ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:05:58.385449 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-761a3f00-039f-4127-a27b-8ee3aec36230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:05:58.394519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 765/1520] 10.222.0.27 () {58 vars in 1058 bytes} [Thu Sep 8 16:05:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:05:58.402462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1bfc9-b775-4e7d-a7bd-84754dd9c307 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:05:58.416434 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1bfc9-b775-4e7d-a7bd-84754dd9c307 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:05:58.421999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46d264b-83d7-4509-8928-44a3560cf141 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd28bb31b55e45668212c9ab69ed3ac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:05:58.422588 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d264b-83d7-4509-8928-44a3560cf141 service placement] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:05:58.422862 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d264b-83d7-4509-8928-44a3560cf141 service placement] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:05:58.423128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46d264b-83d7-4509-8928-44a3560cf141 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:05:58.431835 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d264b-83d7-4509-8928-44a3560cf141 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:05:58.433426 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46d264b-83d7-4509-8928-44a3560cf141 service placement] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:05:58.443591 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 756/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:05:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:06:05.790318 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d617087-954d-495b-9d62-eedea22c5d87 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:05.790935 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d617087-954d-495b-9d62-eedea22c5d87 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:05.790935 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d617087-954d-495b-9d62-eedea22c5d87 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:05.792244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 16:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:06:05.799523 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6cfa8-4744-4b10-8b8d-6644df8ba236 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:05.800283 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6cfa8-4744-4b10-8b8d-6644df8ba236 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:05.800471 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6cfa8-4744-4b10-8b8d-6644df8ba236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:05.830347 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-1ba6cfa8-4744-4b10-8b8d-6644df8ba236 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:06:05.831606 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1ba6cfa8-4744-4b10-8b8d-6644df8ba236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:06:05.840087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 16:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:06:05.999686 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d617087-954d-495b-9d62-eedea22c5d87 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:06:06.014894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d617087-954d-495b-9d62-eedea22c5d87 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:06:06.021802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61261f1d-12be-4c4d-a3cb-2a0f0fc5acaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:06:06.022418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61261f1d-12be-4c4d-a3cb-2a0f0fc5acaa service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:06.022693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61261f1d-12be-4c4d-a3cb-2a0f0fc5acaa service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:06.022967 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61261f1d-12be-4c4d-a3cb-2a0f0fc5acaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:06.032298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61261f1d-12be-4c4d-a3cb-2a0f0fc5acaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:06:06.033922 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61261f1d-12be-4c4d-a3cb-2a0f0fc5acaa service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:06:06.044202 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 767/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:06:07.111080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b433d87-1828-49e5-8dff-08d591bdbd28 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:07.111725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b433d87-1828-49e5-8dff-08d591bdbd28 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:07.111725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b433d87-1828-49e5-8dff-08d591bdbd28 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:07.113015 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 16:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:06:07.120261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa4ff6-9d33-49d0-ad9f-3da6cd507fb0 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:07.121001 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa4ff6-9d33-49d0-ad9f-3da6cd507fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:07.121170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa4ff6-9d33-49d0-ad9f-3da6cd507fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:07.145597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-dbfa4ff6-9d33-49d0-ad9f-3da6cd507fb0 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:06:07.146805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-dbfa4ff6-9d33-49d0-ad9f-3da6cd507fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:06:07.156684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 16:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:06:07.311542 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b433d87-1828-49e5-8dff-08d591bdbd28 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:06:07.328659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b433d87-1828-49e5-8dff-08d591bdbd28 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:06:07.334601 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2484ba-8149-431f-9ea9-3e02fe4f0649 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:06:07.335192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2484ba-8149-431f-9ea9-3e02fe4f0649 service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:07.335478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2484ba-8149-431f-9ea9-3e02fe4f0649 service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:07.335769 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2484ba-8149-431f-9ea9-3e02fe4f0649 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:07.343941 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2484ba-8149-431f-9ea9-3e02fe4f0649 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:06:07.345517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2484ba-8149-431f-9ea9-3e02fe4f0649 service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:06:07.354856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 759/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:06:11.390429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570be7e-ff14-4743-aef5-adc2ab266ff6 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:11.391356 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570be7e-ff14-4743-aef5-adc2ab266ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:11.391356 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570be7e-ff14-4743-aef5-adc2ab266ff6 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:11.393509 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 16:06:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:06:11.401038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6fea8-e26f-465e-806f-a273050e6a5d None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:11.401521 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6fea8-e26f-465e-806f-a273050e6a5d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:11.401521 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b6fea8-e26f-465e-806f-a273050e6a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:11.421505 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-20b6fea8-e26f-465e-806f-a273050e6a5d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:06:11.422694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-20b6fea8-e26f-465e-806f-a273050e6a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:06:11.430758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 16:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:06:11.723291 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570be7e-ff14-4743-aef5-adc2ab266ff6 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:06:11.725470 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a570be7e-ff14-4743-aef5-adc2ab266ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:06:11.808139 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570be7e-ff14-4743-aef5-adc2ab266ff6 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:06:11.812592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3f09ed-6892-4422-b42a-395d7307325c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:06:11.813210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f09ed-6892-4422-b42a-395d7307325c service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:11.813482 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f09ed-6892-4422-b42a-395d7307325c service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:11.813772 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3f09ed-6892-4422-b42a-395d7307325c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:11.824875 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f09ed-6892-4422-b42a-395d7307325c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:06:11.826390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3f09ed-6892-4422-b42a-395d7307325c service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:06:11.835852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 770/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:06:13.956365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32072f42-a750-45b5-8b92-b27fcbcbeed4 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:13.956954 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32072f42-a750-45b5-8b92-b27fcbcbeed4 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:13.956954 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32072f42-a750-45b5-8b92-b27fcbcbeed4 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:13.959101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 16:06:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:06:13.968157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b286b-213d-40f1-b2e8-320a0d3bd632 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:13.968597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b286b-213d-40f1-b2e8-320a0d3bd632 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:13.968749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b286b-213d-40f1-b2e8-320a0d3bd632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:13.995832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-e86b286b-213d-40f1-b2e8-320a0d3bd632 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:06:13.997197 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e86b286b-213d-40f1-b2e8-320a0d3bd632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:06:14.008041 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 16:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:06:14.295621 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32072f42-a750-45b5-8b92-b27fcbcbeed4 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:06:14.307804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32072f42-a750-45b5-8b92-b27fcbcbeed4 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:06:14.311768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313cc12-170b-42d2-8470-3a2c6710f1de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:06:14.312369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313cc12-170b-42d2-8470-3a2c6710f1de service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:14.312643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313cc12-170b-42d2-8470-3a2c6710f1de service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:14.312908 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313cc12-170b-42d2-8470-3a2c6710f1de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:14.321951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313cc12-170b-42d2-8470-3a2c6710f1de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:06:14.323384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313cc12-170b-42d2-8470-3a2c6710f1de service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:06:14.333255 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 762/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:06:18.033366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e58f7-f7c8-4630-950a-62d47af0a7db None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:18.034347 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e58f7-f7c8-4630-950a-62d47af0a7db None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:18.034347 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e58f7-f7c8-4630-950a-62d47af0a7db None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:18.036413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 16:06:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:06:18.041984 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671b348-10e6-4168-9869-44d98bd65f17 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:18.042432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671b348-10e6-4168-9869-44d98bd65f17 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:18.042539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2671b348-10e6-4168-9869-44d98bd65f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:18.050995 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=81 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:06:18.068069 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-2671b348-10e6-4168-9869-44d98bd65f17 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:06:18.069274 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-2671b348-10e6-4168-9869-44d98bd65f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:06:18.077950 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 16:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:06:18.392109 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e58f7-f7c8-4630-950a-62d47af0a7db None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:06:18.394125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3e9e58f7-f7c8-4630-950a-62d47af0a7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:06:18.473606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:06:18.487530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e58f7-f7c8-4630-950a-62d47af0a7db None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:06:18.491332 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e9b8b9-e01b-4a62-bd75-c2cf9ee8ee7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:06:18.491971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9b8b9-e01b-4a62-bd75-c2cf9ee8ee7b service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:18.492246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9b8b9-e01b-4a62-bd75-c2cf9ee8ee7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:18.492550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e9b8b9-e01b-4a62-bd75-c2cf9ee8ee7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:18.502650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9b8b9-e01b-4a62-bd75-c2cf9ee8ee7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:06:18.504171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e9b8b9-e01b-4a62-bd75-c2cf9ee8ee7b service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:06:18.514519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 773/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:06:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:06:20.649956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb4cc1-744a-4ecf-a8c2-54c4cecb57fc None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:20.650527 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb4cc1-744a-4ecf-a8c2-54c4cecb57fc None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:20.650527 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eb4cc1-744a-4ecf-a8c2-54c4cecb57fc None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:20.651917 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 16:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:06:20.658451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029de3eb-0ee2-417a-9118-9d7425e05cd3 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:20.659155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029de3eb-0ee2-417a-9118-9d7425e05cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:20.659357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029de3eb-0ee2-417a-9118-9d7425e05cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:20.684376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-029de3eb-0ee2-417a-9118-9d7425e05cd3 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:06:20.685577 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-029de3eb-0ee2-417a-9118-9d7425e05cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:06:20.694910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 16:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:06:20.997321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb4cc1-744a-4ecf-a8c2-54c4cecb57fc None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:06:20.999250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-94eb4cc1-744a-4ecf-a8c2-54c4cecb57fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:06:21.014219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eb4cc1-744a-4ecf-a8c2-54c4cecb57fc None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:06:21.018672 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9cbae7-724e-4202-9b03-39d14574e592 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:06:21.019269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cbae7-724e-4202-9b03-39d14574e592 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:06:21.019566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cbae7-724e-4202-9b03-39d14574e592 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:06:21.019832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9cbae7-724e-4202-9b03-39d14574e592 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:06:21.029172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9cbae7-724e-4202-9b03-39d14574e592 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:06:21.030566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9cbae7-724e-4202-9b03-39d14574e592 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:06:21.039997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 765/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:06:30.679240 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=11 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:06:31.036018 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=8 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:44.189661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245979e-3fc5-4ae8-a6e7-5d51cc6cbf18 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.190316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245979e-3fc5-4ae8-a6e7-5d51cc6cbf18 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.190316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245979e-3fc5-4ae8-a6e7-5d51cc6cbf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.219556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-a245979e-3fc5-4ae8-a6e7-5d51cc6cbf18 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.220856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a245979e-3fc5-4ae8-a6e7-5d51cc6cbf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.233436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 775/1540] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:44.242920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec8ca3-a53d-4ba5-8074-f1b117af3a17 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.243584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec8ca3-a53d-4ba5-8074-f1b117af3a17 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.243584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdec8ca3-a53d-4ba5-8074-f1b117af3a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.270791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-bdec8ca3-a53d-4ba5-8074-f1b117af3a17 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.272033 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-bdec8ca3-a53d-4ba5-8074-f1b117af3a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.284912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 766/1541] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:44.290440 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245979e-3fc5-4ae8-a6e7-5d51cc6cbf18 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:44.302945 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245979e-3fc5-4ae8-a6e7-5d51cc6cbf18 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:44.312427 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fd93c-72c1-4f65-8fb6-eafb7b086949 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.312751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fd93c-72c1-4f65-8fb6-eafb7b086949 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.312958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6fd93c-72c1-4f65-8fb6-eafb7b086949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.317016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811fbba-4c3c-480b-961a-3f6ff1ce6c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:44.318479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811fbba-4c3c-480b-961a-3f6ff1ce6c75 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.319373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811fbba-4c3c-480b-961a-3f6ff1ce6c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.319669 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811fbba-4c3c-480b-961a-3f6ff1ce6c75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.320327 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811fbba-4c3c-480b-961a-3f6ff1ce6c75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:07:44.320626 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811fbba-4c3c-480b-961a-3f6ff1ce6c75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:44.323588 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811fbba-4c3c-480b-961a-3f6ff1ce6c75 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:44.361572 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-aa6fd93c-72c1-4f65-8fb6-eafb7b086949 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.362791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-aa6fd93c-72c1-4f65-8fb6-eafb7b086949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.364367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 776/1542] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:07:44.372396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811fbba-4c3c-480b-961a-3f6ff1ce6c75 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:44.392407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811fbba-4c3c-480b-961a-3f6ff1ce6c75 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:44.394213 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 767/1543] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:07:44.395512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6844dd3-4654-4ee5-ab5f-5505ecf19e1e None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.395794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6844dd3-4654-4ee5-ab5f-5505ecf19e1e None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.396063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6844dd3-4654-4ee5-ab5f-5505ecf19e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.404752 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f354f-7d63-45aa-ad24-cc1f16c2f296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:44.405377 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f354f-7d63-45aa-ad24-cc1f16c2f296 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.405651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f354f-7d63-45aa-ad24-cc1f16c2f296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.405926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f354f-7d63-45aa-ad24-cc1f16c2f296 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.406563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f354f-7d63-45aa-ad24-cc1f16c2f296 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:44.407610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: WARNING py.warnings [None req-fb2f354f-7d63-45aa-ad24-cc1f16c2f296 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 16:07:44.407610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: warnings.warn(msg) Sep 08 16:07:44.407610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: Sep 08 16:07:44.408801 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f354f-7d63-45aa-ad24-cc1f16c2f296 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:44.432755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-d6844dd3-4654-4ee5-ab5f-5505ecf19e1e None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.439586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d6844dd3-4654-4ee5-ab5f-5505ecf19e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.459252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 768/1544] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:44.460353 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cb324-cb18-4129-802f-24099ece8141 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.460625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cb324-cb18-4129-802f-24099ece8141 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.460894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959cb324-cb18-4129-802f-24099ece8141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.506259 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-959cb324-cb18-4129-802f-24099ece8141 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.508416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-959cb324-cb18-4129-802f-24099ece8141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.528645 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 777/1545] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/projects => generated 405 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:07:44.529991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870a782-0dc4-44e1-96ea-2d01cc6deb5b None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.530295 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870a782-0dc4-44e1-96ea-2d01cc6deb5b None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.530529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b870a782-0dc4-44e1-96ea-2d01cc6deb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.540110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 769/1546] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:44.542796 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd026e44-54b6-42e9-aaf3-df5f2196d6a0 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.543368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd026e44-54b6-42e9-aaf3-df5f2196d6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.543672 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd026e44-54b6-42e9-aaf3-df5f2196d6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.562172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-b870a782-0dc4-44e1-96ea-2d01cc6deb5b None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.563508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b870a782-0dc4-44e1-96ea-2d01cc6deb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.573150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-bd026e44-54b6-42e9-aaf3-df5f2196d6a0 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.574714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-bd026e44-54b6-42e9-aaf3-df5f2196d6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.646815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 778/1547] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:44.646815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962ba28-567f-48b1-a5d2-ab61a5c04aa1 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.646815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962ba28-567f-48b1-a5d2-ab61a5c04aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.646815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962ba28-567f-48b1-a5d2-ab61a5c04aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.666099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 770/1548] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:44.668790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7757c2-faab-4615-b7f7-030ef57c25ef None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.668790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7757c2-faab-4615-b7f7-030ef57c25ef None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.668790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7757c2-faab-4615-b7f7-030ef57c25ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.674104 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-d962ba28-567f-48b1-a5d2-ab61a5c04aa1 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.676922 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d962ba28-567f-48b1-a5d2-ab61a5c04aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.688639 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-9e7757c2-faab-4615-b7f7-030ef57c25ef None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.689745 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 779/1549] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:44.689892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9e7757c2-faab-4615-b7f7-030ef57c25ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.690493 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962ba28-567f-48b1-a5d2-ab61a5c04aa1 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:44.700938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962ba28-567f-48b1-a5d2-ab61a5c04aa1 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:44.705094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 771/1550] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:44.705386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0f386-b841-4cd0-92bb-0812e0abd2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:44.705927 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7757c2-faab-4615-b7f7-030ef57c25ef None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:44.706048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0f386-b841-4cd0-92bb-0812e0abd2df admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.706248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0f386-b841-4cd0-92bb-0812e0abd2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.706501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0f386-b841-4cd0-92bb-0812e0abd2df admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.707127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0f386-b841-4cd0-92bb-0812e0abd2df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:44.709196 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb0f386-b841-4cd0-92bb-0812e0abd2df admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:44.722017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7757c2-faab-4615-b7f7-030ef57c25ef None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:44.727694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7246ec-8cfd-4924-9ef8-862ae2ec1f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:44.728383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7246ec-8cfd-4924-9ef8-862ae2ec1f65 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.728678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7246ec-8cfd-4924-9ef8-862ae2ec1f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.728978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7246ec-8cfd-4924-9ef8-862ae2ec1f65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.729712 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7246ec-8cfd-4924-9ef8-862ae2ec1f65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:07:44.730053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7246ec-8cfd-4924-9ef8-862ae2ec1f65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:44.731363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: WARNING py.warnings [None req-5c7246ec-8cfd-4924-9ef8-862ae2ec1f65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 16:07:44.731363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: warnings.warn(msg) Sep 08 16:07:44.731363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: Sep 08 16:07:44.732446 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7246ec-8cfd-4924-9ef8-862ae2ec1f65 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:44.735182 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 780/1551] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/users => generated 303 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:44.736538 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee66978-0747-4881-9d84-92a0439004de None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.736828 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee66978-0747-4881-9d84-92a0439004de None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.737101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee66978-0747-4881-9d84-92a0439004de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.745558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 772/1552] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:07:44.747734 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de21c2-5b2c-43e1-958e-bf9e39825d05 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.748113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de21c2-5b2c-43e1-958e-bf9e39825d05 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.748583 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de21c2-5b2c-43e1-958e-bf9e39825d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.757183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-aee66978-0747-4881-9d84-92a0439004de None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.758394 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-aee66978-0747-4881-9d84-92a0439004de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.767196 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 781/1553] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:44.767924 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c50e9d-e9c2-427e-ba46-5ef0898a720d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.768187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c50e9d-e9c2-427e-ba46-5ef0898a720d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.768456 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c50e9d-e9c2-427e-ba46-5ef0898a720d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.768794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-90de21c2-5b2c-43e1-958e-bf9e39825d05 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.769960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-90de21c2-5b2c-43e1-958e-bf9e39825d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.779970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 773/1554] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:44.780763 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de21c2-5b2c-43e1-958e-bf9e39825d05 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:44.788061 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-76c50e9d-e9c2-427e-ba46-5ef0898a720d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.789281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-76c50e9d-e9c2-427e-ba46-5ef0898a720d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.790410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de21c2-5b2c-43e1-958e-bf9e39825d05 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:44.795243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac1647-4456-4198-9064-b7ad8a63a9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:44.795936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac1647-4456-4198-9064-b7ad8a63a9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.796178 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac1647-4456-4198-9064-b7ad8a63a9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.796443 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac1647-4456-4198-9064-b7ad8a63a9a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.797103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac1647-4456-4198-9064-b7ad8a63a9a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:07:44.797244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 782/1555] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:44.797404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac1647-4456-4198-9064-b7ad8a63a9a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:44.798266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f95874-217b-4d54-9064-93fd26591e86 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.798520 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f95874-217b-4d54-9064-93fd26591e86 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.798791 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f95874-217b-4d54-9064-93fd26591e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.799002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac1647-4456-4198-9064-b7ad8a63a9a1 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:44.818396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 774/1556] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:07:44.818396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a4beb-c020-41d8-a79c-3e09ed3f104a None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.818396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a4beb-c020-41d8-a79c-3e09ed3f104a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.818396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a4beb-c020-41d8-a79c-3e09ed3f104a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.826465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-80f95874-217b-4d54-9064-93fd26591e86 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.827706 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-80f95874-217b-4d54-9064-93fd26591e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.832970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-936a4beb-c020-41d8-a79c-3e09ed3f104a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.834147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-936a4beb-c020-41d8-a79c-3e09ed3f104a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.836101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 783/1557] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:44.836974 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f95874-217b-4d54-9064-93fd26591e86 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:44.842499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 775/1558] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:44.844236 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a4beb-c020-41d8-a79c-3e09ed3f104a None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:44.847239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f95874-217b-4d54-9064-93fd26591e86 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:44.852186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d1865-846f-4678-bebe-42d0595fedfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:44.852789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d1865-846f-4678-bebe-42d0595fedfd admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.853056 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d1865-846f-4678-bebe-42d0595fedfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.853319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a4beb-c020-41d8-a79c-3e09ed3f104a None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:44.853454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d1865-846f-4678-bebe-42d0595fedfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.853976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d1865-846f-4678-bebe-42d0595fedfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:44.855674 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d1865-846f-4678-bebe-42d0595fedfd admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:44.857855 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6ef71-d041-449f-b406-d950d96e4641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:44.858443 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6ef71-d041-449f-b406-d950d96e4641 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.858715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6ef71-d041-449f-b406-d950d96e4641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.858980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6ef71-d041-449f-b406-d950d96e4641 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.859628 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6ef71-d041-449f-b406-d950d96e4641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:44.861530 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe6ef71-d041-449f-b406-d950d96e4641 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:44.865830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 784/1559] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:44.868833 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e219114-b6b8-42ca-8e59-8824aefba602 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.869175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e219114-b6b8-42ca-8e59-8824aefba602 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.869508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e219114-b6b8-42ca-8e59-8824aefba602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.889520 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-6e219114-b6b8-42ca-8e59-8824aefba602 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.889737 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 776/1560] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:44.890689 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6ef71-d041-449f-b406-d950d96e4641 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:44.890971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6e219114-b6b8-42ca-8e59-8824aefba602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.942776 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:44.956086 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 785/1561] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:44.956231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6ef71-d041-449f-b406-d950d96e4641 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:44.957312 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1b714-7a8f-4804-8845-3bb5fcc4df2d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.957592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1b714-7a8f-4804-8845-3bb5fcc4df2d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.957859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1b714-7a8f-4804-8845-3bb5fcc4df2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.962960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e44791-9a5a-4102-9e42-dcab00cfc3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:44.963612 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e44791-9a5a-4102-9e42-dcab00cfc3e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.963893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e44791-9a5a-4102-9e42-dcab00cfc3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.964207 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e44791-9a5a-4102-9e42-dcab00cfc3e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.964861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e44791-9a5a-4102-9e42-dcab00cfc3e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:07:44.965205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e44791-9a5a-4102-9e42-dcab00cfc3e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:44.966809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e44791-9a5a-4102-9e42-dcab00cfc3e2 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:44.975742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:44.977326 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-b5d1b714-7a8f-4804-8845-3bb5fcc4df2d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:44.978636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b5d1b714-7a8f-4804-8845-3bb5fcc4df2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:44.986458 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 777/1562] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:07:44.986458 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e44791-9a5a-4102-9e42-dcab00cfc3e2 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:44.987149 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 786/1563] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:44.988553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1b714-7a8f-4804-8845-3bb5fcc4df2d None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:44.991707 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e44791-9a5a-4102-9e42-dcab00cfc3e2 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:44.996959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369126f1-296d-46ce-9cdb-ef459ed229d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:44.997561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369126f1-296d-46ce-9cdb-ef459ed229d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:44.997793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1b714-7a8f-4804-8845-3bb5fcc4df2d None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:44.997912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369126f1-296d-46ce-9cdb-ef459ed229d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:44.998087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369126f1-296d-46ce-9cdb-ef459ed229d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:44.998710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369126f1-296d-46ce-9cdb-ef459ed229d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.000646 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369126f1-296d-46ce-9cdb-ef459ed229d2 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.002362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383bd2d-9326-4be9-9e5a-25105aa9b4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.003006 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383bd2d-9326-4be9-9e5a-25105aa9b4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.003223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383bd2d-9326-4be9-9e5a-25105aa9b4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.003515 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383bd2d-9326-4be9-9e5a-25105aa9b4c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.004157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383bd2d-9326-4be9-9e5a-25105aa9b4c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:07:45.004450 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383bd2d-9326-4be9-9e5a-25105aa9b4c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.006032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383bd2d-9326-4be9-9e5a-25105aa9b4c8 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.017665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 787/1564] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:07:45.022905 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383bd2d-9326-4be9-9e5a-25105aa9b4c8 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.028848 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383bd2d-9326-4be9-9e5a-25105aa9b4c8 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.033726 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ddd171-596d-48b3-aab7-f1834c61cc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.034363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ddd171-596d-48b3-aab7-f1834c61cc66 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.034637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ddd171-596d-48b3-aab7-f1834c61cc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.034907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ddd171-596d-48b3-aab7-f1834c61cc66 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.038780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ddd171-596d-48b3-aab7-f1834c61cc66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:07:45.038925 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 778/1565] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:07:44 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:45.039050 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ddd171-596d-48b3-aab7-f1834c61cc66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.039236 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369126f1-296d-46ce-9cdb-ef459ed229d2 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.039345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ddd171-596d-48b3-aab7-f1834c61cc66 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.050429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 788/1566] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:07:45.051152 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ddd171-596d-48b3-aab7-f1834c61cc66 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.060847 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ddd171-596d-48b3-aab7-f1834c61cc66 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.099808 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369126f1-296d-46ce-9cdb-ef459ed229d2 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.104543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25e7ef-7573-425c-b367-35597b781b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.105143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25e7ef-7573-425c-b367-35597b781b08 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.105425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25e7ef-7573-425c-b367-35597b781b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.105730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25e7ef-7573-425c-b367-35597b781b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.106399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25e7ef-7573-425c-b367-35597b781b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.108040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25e7ef-7573-425c-b367-35597b781b08 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.116997 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a0e79-ec3f-4bd1-a732-8377d3db6b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.117602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a0e79-ec3f-4bd1-a732-8377d3db6b3b admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.117870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a0e79-ec3f-4bd1-a732-8377d3db6b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.118134 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a0e79-ec3f-4bd1-a732-8377d3db6b3b admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.118565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 779/1567] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:45.118758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a0e79-ec3f-4bd1-a732-8377d3db6b3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.119374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25e7ef-7573-425c-b367-35597b781b08 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.120692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a0e79-ec3f-4bd1-a732-8377d3db6b3b admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.129173 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25e7ef-7573-425c-b367-35597b781b08 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.136253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9673b-a580-4bd3-892c-dd375c8f419d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.137119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9673b-a580-4bd3-892c-dd375c8f419d admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.137631 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9673b-a580-4bd3-892c-dd375c8f419d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.138021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9673b-a580-4bd3-892c-dd375c8f419d admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.138848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9673b-a580-4bd3-892c-dd375c8f419d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:07:45.139247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9673b-a580-4bd3-892c-dd375c8f419d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.140970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed9673b-a580-4bd3-892c-dd375c8f419d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.145837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 789/1568] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:07:45 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:45.145837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a0e79-ec3f-4bd1-a732-8377d3db6b3b admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.153699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 780/1569] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:07:45.154588 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9673b-a580-4bd3-892c-dd375c8f419d admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.156727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a0e79-ec3f-4bd1-a732-8377d3db6b3b admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.161390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4d23d-6cd1-45ad-9dc6-0baab33f69b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.162128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4d23d-6cd1-45ad-9dc6-0baab33f69b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.162841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4d23d-6cd1-45ad-9dc6-0baab33f69b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.162841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4d23d-6cd1-45ad-9dc6-0baab33f69b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.163292 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4d23d-6cd1-45ad-9dc6-0baab33f69b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.165257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4d23d-6cd1-45ad-9dc6-0baab33f69b5 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.166175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9673b-a580-4bd3-892c-dd375c8f419d admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.173904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768fe65-9361-440d-880d-6ed3737ca601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.174517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768fe65-9361-440d-880d-6ed3737ca601 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.174797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768fe65-9361-440d-880d-6ed3737ca601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.175057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2768fe65-9361-440d-880d-6ed3737ca601 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.175735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768fe65-9361-440d-880d-6ed3737ca601 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:07:45.176032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768fe65-9361-440d-880d-6ed3737ca601 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.177706 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2768fe65-9361-440d-880d-6ed3737ca601 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.190955 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 781/1570] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:07:45.192070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768fe65-9361-440d-880d-6ed3737ca601 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.198283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 790/1571] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:07:45 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:45.199764 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4d23d-6cd1-45ad-9dc6-0baab33f69b5 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.202298 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2768fe65-9361-440d-880d-6ed3737ca601 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.258828 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447500ca-69d5-4666-835e-153122b4dd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.259593 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447500ca-69d5-4666-835e-153122b4dd10 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.259845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447500ca-69d5-4666-835e-153122b4dd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.260217 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447500ca-69d5-4666-835e-153122b4dd10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.260853 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447500ca-69d5-4666-835e-153122b4dd10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.261727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4d23d-6cd1-45ad-9dc6-0baab33f69b5 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.262861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447500ca-69d5-4666-835e-153122b4dd10 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.266327 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a0348-b779-48c0-a20e-2ea4d5bccd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.266937 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a0348-b779-48c0-a20e-2ea4d5bccd9b admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.267213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a0348-b779-48c0-a20e-2ea4d5bccd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.267507 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a0348-b779-48c0-a20e-2ea4d5bccd9b admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.268141 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a0348-b779-48c0-a20e-2ea4d5bccd9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.270085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a0348-b779-48c0-a20e-2ea4d5bccd9b admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.294909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 782/1572] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:07:45 2022] POST /identity/v3/projects => generated 391 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:45.295114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 791/1573] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:07:45 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:45.295201 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447500ca-69d5-4666-835e-153122b4dd10 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.295355 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a0348-b779-48c0-a20e-2ea4d5bccd9b admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.304429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a0348-b779-48c0-a20e-2ea4d5bccd9b admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.320477 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:45.328484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:45.357302 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447500ca-69d5-4666-835e-153122b4dd10 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.361615 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d4622-8ebe-445c-8343-e0cd7589c219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.361988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18476c23-b453-46d3-9185-77f3d938edf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.362427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d4622-8ebe-445c-8343-e0cd7589c219 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.362587 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18476c23-b453-46d3-9185-77f3d938edf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.362841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d4622-8ebe-445c-8343-e0cd7589c219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.362948 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18476c23-b453-46d3-9185-77f3d938edf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.363113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18476c23-b453-46d3-9185-77f3d938edf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.363290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8d4622-8ebe-445c-8343-e0cd7589c219 admin admin] PATH_INFO: `/v3/projects/ef161043b44b40999d6b227b58a369b1/users/911d0d25310b4684af29485659a2578f/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.363770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18476c23-b453-46d3-9185-77f3d938edf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.365672 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18476c23-b453-46d3-9185-77f3d938edf9 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.377842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d4622-8ebe-445c-8343-e0cd7589c219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef161043b44b40999d6b227b58a369b1, user_id=911d0d25310b4684af29485659a2578f, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.378887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: WARNING py.warnings [None req-0c8d4622-8ebe-445c-8343-e0cd7589c219 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 16:07:45.378887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: warnings.warn(msg) Sep 08 16:07:45.378887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: Sep 08 16:07:45.380935 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8d4622-8ebe-445c-8343-e0cd7589c219 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.393823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 783/1574] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:07:45 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:45.394772 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18476c23-b453-46d3-9185-77f3d938edf9 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.394898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 792/1575] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:45 2022] PUT /identity/v3/projects/ef161043b44b40999d6b227b58a369b1/users/911d0d25310b4684af29485659a2578f/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:45.395889 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d4622-8ebe-445c-8343-e0cd7589c219 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.459178 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8d4622-8ebe-445c-8343-e0cd7589c219 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.460928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18476c23-b453-46d3-9185-77f3d938edf9 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.463894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e9b67-64d6-41fb-b798-5fa7eda36694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.464515 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e9b67-64d6-41fb-b798-5fa7eda36694 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.464789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e9b67-64d6-41fb-b798-5fa7eda36694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.465059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e9b67-64d6-41fb-b798-5fa7eda36694 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.465692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e9b67-64d6-41fb-b798-5fa7eda36694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.465877 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecd62f-1722-413d-a9d6-15d3793522f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.466515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecd62f-1722-413d-a9d6-15d3793522f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.466794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecd62f-1722-413d-a9d6-15d3793522f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.467067 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fecd62f-1722-413d-a9d6-15d3793522f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.467622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e9b67-64d6-41fb-b798-5fa7eda36694 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.467743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecd62f-1722-413d-a9d6-15d3793522f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.469336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fecd62f-1722-413d-a9d6-15d3793522f6 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.479269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 784/1576] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:45.479269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecd62f-1722-413d-a9d6-15d3793522f6 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.492624 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fecd62f-1722-413d-a9d6-15d3793522f6 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.495207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 793/1577] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:07:45 2022] POST /identity/v3/projects => generated 357 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:45.496498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e9b67-64d6-41fb-b798-5fa7eda36694 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.550670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d8ea7-3c42-4ecd-bf31-4b0576e774ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.551283 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d8ea7-3c42-4ecd-bf31-4b0576e774ef admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.551581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d8ea7-3c42-4ecd-bf31-4b0576e774ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.551845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d8ea7-3c42-4ecd-bf31-4b0576e774ef admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.552502 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d8ea7-3c42-4ecd-bf31-4b0576e774ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.553516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: WARNING py.warnings [None req-e31d8ea7-3c42-4ecd-bf31-4b0576e774ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 16:07:45.553516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: warnings.warn(msg) Sep 08 16:07:45.553516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: Sep 08 16:07:45.554697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d8ea7-3c42-4ecd-bf31-4b0576e774ef admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.558821 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e9b67-64d6-41fb-b798-5fa7eda36694 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.564361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d2f3d-f404-44c0-8da3-3d3bd8236c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.564960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d2f3d-f404-44c0-8da3-3d3bd8236c17 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.565258 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d2f3d-f404-44c0-8da3-3d3bd8236c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.565511 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d2f3d-f404-44c0-8da3-3d3bd8236c17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.566160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d2f3d-f404-44c0-8da3-3d3bd8236c17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.568366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d2f3d-f404-44c0-8da3-3d3bd8236c17 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.578474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 785/1578] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:07:45 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:45.579748 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d8ea7-3c42-4ecd-bf31-4b0576e774ef admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.589766 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d8ea7-3c42-4ecd-bf31-4b0576e774ef admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.594852 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff932e6-2c80-4957-a522-66b1e246268c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.595523 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff932e6-2c80-4957-a522-66b1e246268c admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.595839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff932e6-2c80-4957-a522-66b1e246268c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.596113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff932e6-2c80-4957-a522-66b1e246268c admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.596755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff932e6-2c80-4957-a522-66b1e246268c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.598429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff932e6-2c80-4957-a522-66b1e246268c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.599217 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 794/1579] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:07:45 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:45.603753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d2f3d-f404-44c0-8da3-3d3bd8236c17 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.612739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 786/1580] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:45.613409 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff932e6-2c80-4957-a522-66b1e246268c admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.617133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:45.624002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff932e6-2c80-4957-a522-66b1e246268c admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.654620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:45.669999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d2f3d-f404-44c0-8da3-3d3bd8236c17 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.674673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db185d56-7e86-42f3-9fbd-c04288c1aba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.675287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db185d56-7e86-42f3-9fbd-c04288c1aba6 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.675591 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db185d56-7e86-42f3-9fbd-c04288c1aba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.675861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db185d56-7e86-42f3-9fbd-c04288c1aba6 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.676520 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db185d56-7e86-42f3-9fbd-c04288c1aba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.678617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db185d56-7e86-42f3-9fbd-c04288c1aba6 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.682987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb836-9abd-426f-9665-16a28825a393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.683654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb836-9abd-426f-9665-16a28825a393 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.684088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb836-9abd-426f-9665-16a28825a393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.684368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733fb836-9abd-426f-9665-16a28825a393 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.685009 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fb836-9abd-426f-9665-16a28825a393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.686969 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733fb836-9abd-426f-9665-16a28825a393 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.702274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 795/1581] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:07:45 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:45.704055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0b327-753f-49d0-bb66-cc99dd3c89ae None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.704431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0b327-753f-49d0-bb66-cc99dd3c89ae None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.704797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0b327-753f-49d0-bb66-cc99dd3c89ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.710596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 787/1582] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:07:45 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:45.711249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb836-9abd-426f-9665-16a28825a393 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.722552 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733fb836-9abd-426f-9665-16a28825a393 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.724445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-dce0b327-753f-49d0-bb66-cc99dd3c89ae None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:45.725750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-dce0b327-753f-49d0-bb66-cc99dd3c89ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:45.728213 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8554f-1334-446d-9fc3-8e58d7da1ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.728890 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8554f-1334-446d-9fc3-8e58d7da1ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.729238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8554f-1334-446d-9fc3-8e58d7da1ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.729560 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff8554f-1334-446d-9fc3-8e58d7da1ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.730271 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8554f-1334-446d-9fc3-8e58d7da1ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.732104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff8554f-1334-446d-9fc3-8e58d7da1ab2 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.733910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 796/1583] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:45.735149 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0b327-753f-49d0-bb66-cc99dd3c89ae None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.742733 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 788/1584] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:45.744679 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8554f-1334-446d-9fc3-8e58d7da1ab2 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.753791 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0b327-753f-49d0-bb66-cc99dd3c89ae None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.754590 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff8554f-1334-446d-9fc3-8e58d7da1ab2 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.759027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7aecc-2f1b-4775-9dd5-d2fd3bd3ae6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.759208 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177c16b-9f80-44ac-9997-068d7edfa111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.759683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7aecc-2f1b-4775-9dd5-d2fd3bd3ae6b admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.759828 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177c16b-9f80-44ac-9997-068d7edfa111 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.759941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7aecc-2f1b-4775-9dd5-d2fd3bd3ae6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.760139 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177c16b-9f80-44ac-9997-068d7edfa111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.760342 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7aecc-2f1b-4775-9dd5-d2fd3bd3ae6b admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.760438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177c16b-9f80-44ac-9997-068d7edfa111 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.760865 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7aecc-2f1b-4775-9dd5-d2fd3bd3ae6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.761034 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177c16b-9f80-44ac-9997-068d7edfa111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.762687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177c16b-9f80-44ac-9997-068d7edfa111 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.762902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7aecc-2f1b-4775-9dd5-d2fd3bd3ae6b admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.773393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 789/1585] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:45.774359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177c16b-9f80-44ac-9997-068d7edfa111 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.789070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 797/1586] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:07:45 2022] POST /identity/v3/users => generated 279 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:45.789272 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177c16b-9f80-44ac-9997-068d7edfa111 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.789392 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7aecc-2f1b-4775-9dd5-d2fd3bd3ae6b admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.793934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991ab3a-43e9-438a-a83e-61aff40147f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.794578 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991ab3a-43e9-438a-a83e-61aff40147f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.794853 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991ab3a-43e9-438a-a83e-61aff40147f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.795125 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991ab3a-43e9-438a-a83e-61aff40147f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.795797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f991ab3a-43e9-438a-a83e-61aff40147f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.796626 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7aecc-2f1b-4775-9dd5-d2fd3bd3ae6b admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.797742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f991ab3a-43e9-438a-a83e-61aff40147f8 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.801299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738dbc9-425c-452d-a01b-d67461754341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.801923 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738dbc9-425c-452d-a01b-d67461754341 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.802205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738dbc9-425c-452d-a01b-d67461754341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.802480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738dbc9-425c-452d-a01b-d67461754341 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.803150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738dbc9-425c-452d-a01b-d67461754341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.804801 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738dbc9-425c-452d-a01b-d67461754341 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.816285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 798/1587] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:45.816285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738dbc9-425c-452d-a01b-d67461754341 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.822965 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 790/1588] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:07:45 2022] POST /identity/v3/users => generated 295 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:45.822965 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991ab3a-43e9-438a-a83e-61aff40147f8 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.828231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738dbc9-425c-452d-a01b-d67461754341 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.830622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991ab3a-43e9-438a-a83e-61aff40147f8 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.832994 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062e21b-0974-4b74-9d35-62584e17c507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.833853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062e21b-0974-4b74-9d35-62584e17c507 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.834079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062e21b-0974-4b74-9d35-62584e17c507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.834417 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062e21b-0974-4b74-9d35-62584e17c507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.835155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31e7ac-6ac7-47ea-8fce-08ed9bae64e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.835314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062e21b-0974-4b74-9d35-62584e17c507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.835769 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31e7ac-6ac7-47ea-8fce-08ed9bae64e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.836045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31e7ac-6ac7-47ea-8fce-08ed9bae64e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.836314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd31e7ac-6ac7-47ea-8fce-08ed9bae64e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.836808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8062e21b-0974-4b74-9d35-62584e17c507 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.836954 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31e7ac-6ac7-47ea-8fce-08ed9bae64e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.838535 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd31e7ac-6ac7-47ea-8fce-08ed9bae64e5 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.847669 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 799/1589] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:45.848478 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062e21b-0974-4b74-9d35-62584e17c507 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.851755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 791/1590] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:45.851755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31e7ac-6ac7-47ea-8fce-08ed9bae64e5 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.853084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8062e21b-0974-4b74-9d35-62584e17c507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:45.860878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd31e7ac-6ac7-47ea-8fce-08ed9bae64e5 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.865590 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ddd3d-55fb-47da-af45-b1371a9b3c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.866241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ddd3d-55fb-47da-af45-b1371a9b3c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.866519 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ddd3d-55fb-47da-af45-b1371a9b3c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.866790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ddd3d-55fb-47da-af45-b1371a9b3c73 admin admin] PATH_INFO: `/v3/projects/e604a196b20448caa9a0f2196b7f308e/users/2ffcc5ea76744215b8a4a7ea49982913/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.881150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ddd3d-55fb-47da-af45-b1371a9b3c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e604a196b20448caa9a0f2196b7f308e, user_id=2ffcc5ea76744215b8a4a7ea49982913, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.883924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ddd3d-55fb-47da-af45-b1371a9b3c73 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.898093 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 792/1591] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:45 2022] PUT /identity/v3/projects/e604a196b20448caa9a0f2196b7f308e/users/2ffcc5ea76744215b8a4a7ea49982913/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:45.899057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ddd3d-55fb-47da-af45-b1371a9b3c73 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.921743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8062e21b-0974-4b74-9d35-62584e17c507 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.963174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ddd3d-55fb-47da-af45-b1371a9b3c73 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:45.968471 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90d7b4-7e46-4f10-8569-a6c90f260f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.969092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90d7b4-7e46-4f10-8569-a6c90f260f6d admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.969368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90d7b4-7e46-4f10-8569-a6c90f260f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.969638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc90d7b4-7e46-4f10-8569-a6c90f260f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.970315 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90d7b4-7e46-4f10-8569-a6c90f260f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.971289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b58a6-d236-4120-a377-8edfca55fa25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:45.971943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b58a6-d236-4120-a377-8edfca55fa25 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:45.972063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc90d7b4-7e46-4f10-8569-a6c90f260f6d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.972221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b58a6-d236-4120-a377-8edfca55fa25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:45.972495 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b58a6-d236-4120-a377-8edfca55fa25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:45.981652 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b58a6-d236-4120-a377-8edfca55fa25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:45.982162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 793/1592] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:45.982982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90d7b4-7e46-4f10-8569-a6c90f260f6d admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.983231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b58a6-d236-4120-a377-8edfca55fa25 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:45.992943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 800/1593] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:07:45.993780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b58a6-d236-4120-a377-8edfca55fa25 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:45.995509 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc90d7b4-7e46-4f10-8569-a6c90f260f6d admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:46.002460 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b40ac5-9c91-4310-8642-233f222b6669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:46.002948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b58a6-d236-4120-a377-8edfca55fa25 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:46.003282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b40ac5-9c91-4310-8642-233f222b6669 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.003696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b40ac5-9c91-4310-8642-233f222b6669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.004059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b40ac5-9c91-4310-8642-233f222b6669 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.004799 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b40ac5-9c91-4310-8642-233f222b6669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:46.006496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b40ac5-9c91-4310-8642-233f222b6669 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:46.008231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505679fb-2399-4f82-8d67-4445f18d6b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:46.008557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505679fb-2399-4f82-8d67-4445f18d6b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.008842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505679fb-2399-4f82-8d67-4445f18d6b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.009118 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505679fb-2399-4f82-8d67-4445f18d6b1e admin admin] PATH_INFO: `/v3/projects/f22c36df7bc547f6abd9533d0730a077/users/44e6d392d538462a875805138980da85/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.019081 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 794/1594] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:46.020635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b40ac5-9c91-4310-8642-233f222b6669 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:46.024510 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:46.029255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505679fb-2399-4f82-8d67-4445f18d6b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f22c36df7bc547f6abd9533d0730a077, user_id=44e6d392d538462a875805138980da85, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:46.031763 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b40ac5-9c91-4310-8642-233f222b6669 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:46.032937 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505679fb-2399-4f82-8d67-4445f18d6b1e admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:46.038259 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bdc63d-a363-45e3-9c10-cb16d2f13f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:46.039000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdc63d-a363-45e3-9c10-cb16d2f13f53 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.039247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdc63d-a363-45e3-9c10-cb16d2f13f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.039554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdc63d-a363-45e3-9c10-cb16d2f13f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.040240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bdc63d-a363-45e3-9c10-cb16d2f13f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:46.041825 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bdc63d-a363-45e3-9c10-cb16d2f13f53 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:46.049311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 801/1595] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:45 2022] PUT /identity/v3/projects/f22c36df7bc547f6abd9533d0730a077/users/44e6d392d538462a875805138980da85/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:46.050515 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505679fb-2399-4f82-8d67-4445f18d6b1e admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:46.066449 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 795/1596] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:46.067537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bdc63d-a363-45e3-9c10-cb16d2f13f53 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:46.078844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bdc63d-a363-45e3-9c10-cb16d2f13f53 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:46.101850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:46.125713 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505679fb-2399-4f82-8d67-4445f18d6b1e admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:46.132423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b1eb05-c80e-4274-aa5e-884fb1a39e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:46.133155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1eb05-c80e-4274-aa5e-884fb1a39e0a admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.133497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1eb05-c80e-4274-aa5e-884fb1a39e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.133909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b1eb05-c80e-4274-aa5e-884fb1a39e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.134684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1eb05-c80e-4274-aa5e-884fb1a39e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:46.136941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b1eb05-c80e-4274-aa5e-884fb1a39e0a admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:46.148418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cb59f-8661-4b6f-bbd7-320d02179e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:46.149024 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 802/1597] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:46.149142 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cb59f-8661-4b6f-bbd7-320d02179e9a admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.149301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cb59f-8661-4b6f-bbd7-320d02179e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.149728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88cb59f-8661-4b6f-bbd7-320d02179e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.150260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb66c1-ce9a-44c5-8d0f-601888241bbf None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.150381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88cb59f-8661-4b6f-bbd7-320d02179e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:46.150594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb66c1-ce9a-44c5-8d0f-601888241bbf None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.150870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb66c1-ce9a-44c5-8d0f-601888241bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.151975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88cb59f-8661-4b6f-bbd7-320d02179e9a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:46.164328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 796/1598] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:46.165188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cb59f-8661-4b6f-bbd7-320d02179e9a admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:46.177187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88cb59f-8661-4b6f-bbd7-320d02179e9a admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:46.188924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054aea8-4f22-44b4-9478-74daaecfac48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:46.189115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-51bb66c1-ce9a-44c5-8d0f-601888241bbf None None] MFA Rules not processed for user `2ffcc5ea76744215b8a4a7ea49982913`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:46.189825 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054aea8-4f22-44b4-9478-74daaecfac48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.190155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-51bb66c1-ce9a-44c5-8d0f-601888241bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:46.190279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054aea8-4f22-44b4-9478-74daaecfac48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.190570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054aea8-4f22-44b4-9478-74daaecfac48 admin admin] PATH_INFO: `/v3/projects/be0b2aab8dbf41a7a3c590ca61f022f8/users/675b1a5759844cdaa7e0bf071a3ee030/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.212146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054aea8-4f22-44b4-9478-74daaecfac48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be0b2aab8dbf41a7a3c590ca61f022f8, user_id=675b1a5759844cdaa7e0bf071a3ee030, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:46.216124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6054aea8-4f22-44b4-9478-74daaecfac48 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:46.232181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 797/1599] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:46 2022] PUT /identity/v3/projects/be0b2aab8dbf41a7a3c590ca61f022f8/users/675b1a5759844cdaa7e0bf071a3ee030/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:46.236780 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054aea8-4f22-44b4-9478-74daaecfac48 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:46.279285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 803/1600] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:46.285706 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be6147-6ea1-4adf-b3ec-433ca5588f14 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.285994 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be6147-6ea1-4adf-b3ec-433ca5588f14 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.286273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be6147-6ea1-4adf-b3ec-433ca5588f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.318464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054aea8-4f22-44b4-9478-74daaecfac48 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:46.320173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f3be6147-6ea1-4adf-b3ec-433ca5588f14 None None] MFA Rules not processed for user `44e6d392d538462a875805138980da85`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:46.321339 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f3be6147-6ea1-4adf-b3ec-433ca5588f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:46.324826 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589ce6f-33ec-44cb-ba10-537d413957f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:46.325616 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589ce6f-33ec-44cb-ba10-537d413957f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.325945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589ce6f-33ec-44cb-ba10-537d413957f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.326232 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589ce6f-33ec-44cb-ba10-537d413957f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.327006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589ce6f-33ec-44cb-ba10-537d413957f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:46.328950 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589ce6f-33ec-44cb-ba10-537d413957f0 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:46.342746 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 798/1601] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:46.343770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589ce6f-33ec-44cb-ba10-537d413957f0 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:46.357355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589ce6f-33ec-44cb-ba10-537d413957f0 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:46.362989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe95875-f4fe-4c90-bb6e-0ad41717bf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:46.367980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe95875-f4fe-4c90-bb6e-0ad41717bf32 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.368557 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe95875-f4fe-4c90-bb6e-0ad41717bf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.369041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe95875-f4fe-4c90-bb6e-0ad41717bf32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.369942 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe95875-f4fe-4c90-bb6e-0ad41717bf32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:46.372499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe95875-f4fe-4c90-bb6e-0ad41717bf32 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:46.388964 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 799/1602] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:46.390394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe95875-f4fe-4c90-bb6e-0ad41717bf32 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:46.405014 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe95875-f4fe-4c90-bb6e-0ad41717bf32 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:46.410341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6ba45-2025-48fa-92b6-9675c2eb4ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:46.411215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6ba45-2025-48fa-92b6-9675c2eb4ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.411587 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6ba45-2025-48fa-92b6-9675c2eb4ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.411870 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6ba45-2025-48fa-92b6-9675c2eb4ac3 admin admin] PATH_INFO: `/v3/projects/73c9cf0f579144eb80c090456d363a05/users/18375266a64348df879a9eec53bfcbf9/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.423669 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 804/1603] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:46.424916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be6147-6ea1-4adf-b3ec-433ca5588f14 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:46.430958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6ba45-2025-48fa-92b6-9675c2eb4ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73c9cf0f579144eb80c090456d363a05, user_id=18375266a64348df879a9eec53bfcbf9, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:46.434063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c6ba45-2025-48fa-92b6-9675c2eb4ac3 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:46.436817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be6147-6ea1-4adf-b3ec-433ca5588f14 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:46.444946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fdde83-855b-4e8c-ac0a-bd57c9b9474a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:46.445775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdde83-855b-4e8c-ac0a-bd57c9b9474a admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.446044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdde83-855b-4e8c-ac0a-bd57c9b9474a admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.446319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdde83-855b-4e8c-ac0a-bd57c9b9474a admin admin] PATH_INFO: `/v3/projects/91730a2601824af19a540c48681e6fc3/users/00b673d5c65d40d9a90cc3b579e83e7f/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.449896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:46.466261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 800/1604] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:46 2022] PUT /identity/v3/projects/73c9cf0f579144eb80c090456d363a05/users/18375266a64348df879a9eec53bfcbf9/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:46.471357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49169ddc-8107-4641-bf2a-a9cbe4f9e200 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.471703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49169ddc-8107-4641-bf2a-a9cbe4f9e200 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.472066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49169ddc-8107-4641-bf2a-a9cbe4f9e200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.476948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdde83-855b-4e8c-ac0a-bd57c9b9474a admin admin] RBAC: Authorizing `identity:create_grant(project_id=91730a2601824af19a540c48681e6fc3, user_id=00b673d5c65d40d9a90cc3b579e83e7f, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:46.481058 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fdde83-855b-4e8c-ac0a-bd57c9b9474a admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:46.499157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 805/1605] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:46 2022] PUT /identity/v3/projects/91730a2601824af19a540c48681e6fc3/users/00b673d5c65d40d9a90cc3b579e83e7f/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:46.499829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-49169ddc-8107-4641-bf2a-a9cbe4f9e200 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:46.500857 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50225466-474a-440c-b3bb-b8cc1508a269 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.501195 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-49169ddc-8107-4641-bf2a-a9cbe4f9e200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:46.501334 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50225466-474a-440c-b3bb-b8cc1508a269 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.501576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50225466-474a-440c-b3bb-b8cc1508a269 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.503009 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 806/1606] 10.222.0.27 () {54 vars in 962 bytes} [Thu Sep 8 16:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:07:46.503564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50225466-474a-440c-b3bb-b8cc1508a269 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:46.564653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:46.578384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 801/1607] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:46.579633 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49169ddc-8107-4641-bf2a-a9cbe4f9e200 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:46.579777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50225466-474a-440c-b3bb-b8cc1508a269 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:46.586431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b9a91-7df4-4c5c-9d36-2c323e82c80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:46.587167 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b9a91-7df4-4c5c-9d36-2c323e82c80d admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.587568 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b9a91-7df4-4c5c-9d36-2c323e82c80d admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.587749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b9a91-7df4-4c5c-9d36-2c323e82c80d admin admin] PATH_INFO: `/v3/projects/a829c94f7acb470c8908b01460e9f070/users/83980d1e62ff463fae278ddd760c59ce/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.592790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49169ddc-8107-4641-bf2a-a9cbe4f9e200 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:46.597886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d789b-f29b-48a5-bde2-80f98a864d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:46.598654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d789b-f29b-48a5-bde2-80f98a864d39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.598926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d789b-f29b-48a5-bde2-80f98a864d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.599199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d789b-f29b-48a5-bde2-80f98a864d39 admin admin] PATH_INFO: `/v3/projects/52651960a8b6494193c5e8abc658a2dd/users/c89859a30a2442b8ae13fe0444782d9c/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.604289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b9a91-7df4-4c5c-9d36-2c323e82c80d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a829c94f7acb470c8908b01460e9f070, user_id=83980d1e62ff463fae278ddd760c59ce, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:46.607468 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b9a91-7df4-4c5c-9d36-2c323e82c80d admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:46.615964 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d789b-f29b-48a5-bde2-80f98a864d39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52651960a8b6494193c5e8abc658a2dd, user_id=c89859a30a2442b8ae13fe0444782d9c, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:46.618623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d789b-f29b-48a5-bde2-80f98a864d39 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:46.623143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 807/1608] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:46 2022] PUT /identity/v3/projects/a829c94f7acb470c8908b01460e9f070/users/83980d1e62ff463fae278ddd760c59ce/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:46.624808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd0bf6-fa9d-4073-9123-7a80fc641477 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.625552 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd0bf6-fa9d-4073-9123-7a80fc641477 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.626092 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccd0bf6-fa9d-4073-9123-7a80fc641477 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.627961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 808/1609] 10.222.0.27 () {54 vars in 962 bytes} [Thu Sep 8 16:07:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:07:46.629172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a43ad-676e-4c3f-a208-77607df60c13 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.629635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a43ad-676e-4c3f-a208-77607df60c13 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.630069 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a43ad-676e-4c3f-a208-77607df60c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.645850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 802/1610] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:46 2022] PUT /identity/v3/projects/52651960a8b6494193c5e8abc658a2dd/users/c89859a30a2442b8ae13fe0444782d9c/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:46.647131 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679041f0-29f9-420c-bc63-740062e153f9 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.647415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679041f0-29f9-420c-bc63-740062e153f9 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.647717 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679041f0-29f9-420c-bc63-740062e153f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.655428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-aa2a43ad-676e-4c3f-a208-77607df60c13 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:46.657652 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-aa2a43ad-676e-4c3f-a208-77607df60c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:46.667787 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-679041f0-29f9-420c-bc63-740062e153f9 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:46.668917 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-679041f0-29f9-420c-bc63-740062e153f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:46.745937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 803/1611] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:46.745937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6d42e-62e5-4f75-a4de-4ae8baba8c66 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.745937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6d42e-62e5-4f75-a4de-4ae8baba8c66 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.745937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6d42e-62e5-4f75-a4de-4ae8baba8c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.765407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 809/1612] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:46.769112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a43ad-676e-4c3f-a208-77607df60c13 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:46.802844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-43c6d42e-62e5-4f75-a4de-4ae8baba8c66 None None] MFA Rules not processed for user `6e9209e77ce44b128e25a8ddcf78f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:46.804470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-43c6d42e-62e5-4f75-a4de-4ae8baba8c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:46.829211 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a43ad-676e-4c3f-a208-77607df60c13 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:46.833459 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350fe24-27bc-4551-999c-b6bc35241b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:46.833746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350fe24-27bc-4551-999c-b6bc35241b8f service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.833955 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350fe24-27bc-4551-999c-b6bc35241b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.834230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350fe24-27bc-4551-999c-b6bc35241b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.845281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350fe24-27bc-4551-999c-b6bc35241b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:46.846825 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350fe24-27bc-4551-999c-b6bc35241b8f service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:46.857498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 810/1613] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:07:46.858831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d7067-595b-44bc-9d40-768485667b7d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.859124 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d7067-595b-44bc-9d40-768485667b7d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.859401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d7067-595b-44bc-9d40-768485667b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.882386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:46.884380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 804/1614] 10.222.0.27 () {58 vars in 1059 bytes} [Thu Sep 8 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:46.886157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd14df-eb46-4583-8834-6b58b4eb80b4 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.886444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd14df-eb46-4583-8834-6b58b4eb80b4 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.886872 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd14df-eb46-4583-8834-6b58b4eb80b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.898030 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-630d7067-595b-44bc-9d40-768485667b7d None None] MFA Rules not processed for user `83980d1e62ff463fae278ddd760c59ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:46.901003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-630d7067-595b-44bc-9d40-768485667b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:46.910804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-f7dd14df-eb46-4583-8834-6b58b4eb80b4 None None] MFA Rules not processed for user `6e9209e77ce44b128e25a8ddcf78f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:46.912121 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f7dd14df-eb46-4583-8834-6b58b4eb80b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:46.923311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 805/1615] 10.222.0.27 () {58 vars in 1059 bytes} [Thu Sep 8 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:46.925531 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b973db-4704-4e24-a302-b281d2a6cf1a None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.925531 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b973db-4704-4e24-a302-b281d2a6cf1a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.925783 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b973db-4704-4e24-a302-b281d2a6cf1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.952249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-85b973db-4704-4e24-a302-b281d2a6cf1a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:46.953750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-85b973db-4704-4e24-a302-b281d2a6cf1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:46.965421 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 806/1616] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:46.971788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ee1d1-903d-47aa-ac5d-fd58919860df None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:46.972037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ee1d1-903d-47aa-ac5d-fd58919860df None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:46.972725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ee1d1-903d-47aa-ac5d-fd58919860df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:46.997549 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 811/1617] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:46.998766 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d7067-595b-44bc-9d40-768485667b7d None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:47.008124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-c83ee1d1-903d-47aa-ac5d-fd58919860df None None] MFA Rules not processed for user `911d0d25310b4684af29485659a2578f`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:47.009340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c83ee1d1-903d-47aa-ac5d-fd58919860df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:47.009506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d7067-595b-44bc-9d40-768485667b7d None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:47.013834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7376d39-eccf-4a5c-8fa2-d89b30e6410a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:47.014475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7376d39-eccf-4a5c-8fa2-d89b30e6410a service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.014752 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7376d39-eccf-4a5c-8fa2-d89b30e6410a service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.015018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7376d39-eccf-4a5c-8fa2-d89b30e6410a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.025780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7376d39-eccf-4a5c-8fa2-d89b30e6410a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:47.027701 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7376d39-eccf-4a5c-8fa2-d89b30e6410a service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:47.039399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 812/1618] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:07:47.040252 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7376d39-eccf-4a5c-8fa2-d89b30e6410a service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:47.043099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:47.051651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7376d39-eccf-4a5c-8fa2-d89b30e6410a service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:47.056283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31152019-4ca8-4b16-8e41-3862464cb3a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:47.056884 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31152019-4ca8-4b16-8e41-3862464cb3a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.057157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31152019-4ca8-4b16-8e41-3862464cb3a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.057426 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31152019-4ca8-4b16-8e41-3862464cb3a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.068074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31152019-4ca8-4b16-8e41-3862464cb3a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:47.069609 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31152019-4ca8-4b16-8e41-3862464cb3a2 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:47.081079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 813/1619] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:07:47.082189 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31152019-4ca8-4b16-8e41-3862464cb3a2 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:47.096791 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31152019-4ca8-4b16-8e41-3862464cb3a2 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:47.098128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 807/1620] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:47.099182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ee1d1-903d-47aa-ac5d-fd58919860df None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:47.105239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1161ed-82ee-43d1-8e00-45b287b0cf3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:47.106084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1161ed-82ee-43d1-8e00-45b287b0cf3f service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.106454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1161ed-82ee-43d1-8e00-45b287b0cf3f service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.106837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1161ed-82ee-43d1-8e00-45b287b0cf3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.131486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ee1d1-903d-47aa-ac5d-fd58919860df None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:47.141441 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17028a4e-10b2-46a3-b2f8-780c27dca530 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:47.141961 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17028a4e-10b2-46a3-b2f8-780c27dca530 service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.142256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17028a4e-10b2-46a3-b2f8-780c27dca530 service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.142591 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17028a4e-10b2-46a3-b2f8-780c27dca530 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.204036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1161ed-82ee-43d1-8e00-45b287b0cf3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:47.209262 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1161ed-82ee-43d1-8e00-45b287b0cf3f service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:47.226891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17028a4e-10b2-46a3-b2f8-780c27dca530 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:47.228950 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17028a4e-10b2-46a3-b2f8-780c27dca530 service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:47.230786 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 814/1621] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:07:47.232540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1161ed-82ee-43d1-8e00-45b287b0cf3f service nova] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:47.242360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 808/1622] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:07:47.243502 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17028a4e-10b2-46a3-b2f8-780c27dca530 service nova] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:47.256396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17028a4e-10b2-46a3-b2f8-780c27dca530 service nova] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:47.261982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37bca39-b332-432b-af0e-ff573e08efe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:47.262917 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37bca39-b332-432b-af0e-ff573e08efe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.263393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37bca39-b332-432b-af0e-ff573e08efe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.263871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37bca39-b332-432b-af0e-ff573e08efe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.264740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37bca39-b332-432b-af0e-ff573e08efe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:47.267036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37bca39-b332-432b-af0e-ff573e08efe6 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:47.267574 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1161ed-82ee-43d1-8e00-45b287b0cf3f service nova] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:47.277270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c2337-3a5d-4c18-b651-ef7705bee361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:47.279159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c2337-3a5d-4c18-b651-ef7705bee361 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.279710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c2337-3a5d-4c18-b651-ef7705bee361 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.280279 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c2337-3a5d-4c18-b651-ef7705bee361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.302516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c2337-3a5d-4c18-b651-ef7705bee361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:47.305372 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c2337-3a5d-4c18-b651-ef7705bee361 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:47.321535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 815/1623] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:07:47.323104 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c2337-3a5d-4c18-b651-ef7705bee361 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:47.333075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 809/1624] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:07:47 2022] POST /identity/v3/projects => generated 353 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:47.335224 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37bca39-b332-432b-af0e-ff573e08efe6 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:47.337379 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c2337-3a5d-4c18-b651-ef7705bee361 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:47.423571 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:47.438165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb34eb8-19b5-4daa-a64e-0a24175cc847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:47.439975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb34eb8-19b5-4daa-a64e-0a24175cc847 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.440418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb34eb8-19b5-4daa-a64e-0a24175cc847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.440810 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb34eb8-19b5-4daa-a64e-0a24175cc847 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.441479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb34eb8-19b5-4daa-a64e-0a24175cc847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:47.446695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb34eb8-19b5-4daa-a64e-0a24175cc847 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:47.457531 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37bca39-b332-432b-af0e-ff573e08efe6 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:47.464298 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba62e2-f773-4b52-aa69-65d3826ffbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:47.464696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba62e2-f773-4b52-aa69-65d3826ffbed admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.465002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba62e2-f773-4b52-aa69-65d3826ffbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.465362 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba62e2-f773-4b52-aa69-65d3826ffbed admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.466009 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba62e2-f773-4b52-aa69-65d3826ffbed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:47.467985 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba62e2-f773-4b52-aa69-65d3826ffbed admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:47.507481 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 816/1625] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:07:47 2022] POST /identity/v3/projects => generated 403 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:47.508101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 810/1626] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:07:47 2022] POST /identity/v3/users => generated 283 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:47.512238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb34eb8-19b5-4daa-a64e-0a24175cc847 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:47.512404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba62e2-f773-4b52-aa69-65d3826ffbed admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:47.540596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba62e2-f773-4b52-aa69-65d3826ffbed admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:47.601267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb34eb8-19b5-4daa-a64e-0a24175cc847 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:47.607929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43d97d-a5d2-480b-b34d-15bacd8001ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:47.608781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43d97d-a5d2-480b-b34d-15bacd8001ab admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.609128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43d97d-a5d2-480b-b34d-15bacd8001ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.609505 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43d97d-a5d2-480b-b34d-15bacd8001ab admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.610275 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43d97d-a5d2-480b-b34d-15bacd8001ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:47.612440 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43d97d-a5d2-480b-b34d-15bacd8001ab admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:47.632542 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:47.656359 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 817/1627] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:07:47 2022] POST /identity/v3/users => generated 308 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:47.656359 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43d97d-a5d2-480b-b34d-15bacd8001ab admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:47.691080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43d97d-a5d2-480b-b34d-15bacd8001ab admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:47.693544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03b3b7-d483-4c28-b712-f57b86a29232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:47.694167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03b3b7-d483-4c28-b712-f57b86a29232 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.694441 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03b3b7-d483-4c28-b712-f57b86a29232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.694710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db03b3b7-d483-4c28-b712-f57b86a29232 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.695645 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03b3b7-d483-4c28-b712-f57b86a29232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:47.698033 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db03b3b7-d483-4c28-b712-f57b86a29232 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:47.709671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f42428-af8c-4d3d-8452-9e210cbf0002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:47.709671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f42428-af8c-4d3d-8452-9e210cbf0002 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.709671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f42428-af8c-4d3d-8452-9e210cbf0002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.709671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f42428-af8c-4d3d-8452-9e210cbf0002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.710259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f42428-af8c-4d3d-8452-9e210cbf0002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:47.714169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f42428-af8c-4d3d-8452-9e210cbf0002 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:47.727235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 811/1628] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:47.732869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03b3b7-d483-4c28-b712-f57b86a29232 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:47.738325 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 818/1629] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:47.739379 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f42428-af8c-4d3d-8452-9e210cbf0002 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:47.756247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f42428-af8c-4d3d-8452-9e210cbf0002 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:47.757124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db03b3b7-d483-4c28-b712-f57b86a29232 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:47.772099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55a558-394c-4822-83ec-286f7b0ff1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:47.773718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188a1d6-d949-4aa2-84ff-dd3d7dbd304e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:47.773861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55a558-394c-4822-83ec-286f7b0ff1c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.774084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55a558-394c-4822-83ec-286f7b0ff1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.774296 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55a558-394c-4822-83ec-286f7b0ff1c0 admin admin] PATH_INFO: `/v3/projects/8c6c243829fe4865b0513fdaec55d0b8/users/701782078ed44f0fb89be2f58edac4db/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.774407 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188a1d6-d949-4aa2-84ff-dd3d7dbd304e admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.774999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188a1d6-d949-4aa2-84ff-dd3d7dbd304e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.774999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188a1d6-d949-4aa2-84ff-dd3d7dbd304e admin admin] PATH_INFO: `/v3/projects/45a000f6a7cf4acf80da045a99630f91/users/eb84d447806249a48f85a563c3183aee/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.805438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188a1d6-d949-4aa2-84ff-dd3d7dbd304e admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a000f6a7cf4acf80da045a99630f91, user_id=eb84d447806249a48f85a563c3183aee, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:47.808050 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188a1d6-d949-4aa2-84ff-dd3d7dbd304e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:47.810493 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55a558-394c-4822-83ec-286f7b0ff1c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c6c243829fe4865b0513fdaec55d0b8, user_id=701782078ed44f0fb89be2f58edac4db, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:47.814208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c55a558-394c-4822-83ec-286f7b0ff1c0 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:47.838650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 812/1630] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:47 2022] PUT /identity/v3/projects/45a000f6a7cf4acf80da045a99630f91/users/eb84d447806249a48f85a563c3183aee/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:47.842949 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde163f7-4f6c-43c9-945e-e6032dedab68 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.843225 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde163f7-4f6c-43c9-945e-e6032dedab68 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.843527 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde163f7-4f6c-43c9-945e-e6032dedab68 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.850626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 813/1632] 10.222.0.27 () {54 vars in 912 bytes} [Thu Sep 8 16:07:47 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:07:47.852214 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde163f7-4f6c-43c9-945e-e6032dedab68 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:47.858203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 819/1632] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:47 2022] PUT /identity/v3/projects/8c6c243829fe4865b0513fdaec55d0b8/users/701782078ed44f0fb89be2f58edac4db/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:47.859339 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55a558-394c-4822-83ec-286f7b0ff1c0 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:47.978176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55a558-394c-4822-83ec-286f7b0ff1c0 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:47.982099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde163f7-4f6c-43c9-945e-e6032dedab68 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:47.988174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b86672-8092-4cdb-a605-73bf333e8d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:47.988739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b86672-8092-4cdb-a605-73bf333e8d0c admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.989011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b86672-8092-4cdb-a605-73bf333e8d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.989926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b86672-8092-4cdb-a605-73bf333e8d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.990582 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b86672-8092-4cdb-a605-73bf333e8d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:47.992244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b86672-8092-4cdb-a605-73bf333e8d0c admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:47.993735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159912ae-3b6d-4318-8c0f-a3fc1f2c8174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:47.994350 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159912ae-3b6d-4318-8c0f-a3fc1f2c8174 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:47.994620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159912ae-3b6d-4318-8c0f-a3fc1f2c8174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:47.994885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159912ae-3b6d-4318-8c0f-a3fc1f2c8174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:47.995669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159912ae-3b6d-4318-8c0f-a3fc1f2c8174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:47.997320 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159912ae-3b6d-4318-8c0f-a3fc1f2c8174 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:48.017630 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 820/1633] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:48.018955 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316b1a9-8bf6-4806-a06f-455010a17b94 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:48.019238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316b1a9-8bf6-4806-a06f-455010a17b94 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:48.019535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316b1a9-8bf6-4806-a06f-455010a17b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:48.023799 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 814/1634] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:48.029026 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9ba21-0083-48d4-9dc8-46393ce5e605 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:48.029307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9ba21-0083-48d4-9dc8-46393ce5e605 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:48.029569 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9ba21-0083-48d4-9dc8-46393ce5e605 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:48.031063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 815/1635] 10.222.0.27 () {54 vars in 912 bytes} [Thu Sep 8 16:07:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:07:48.036384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9ba21-0083-48d4-9dc8-46393ce5e605 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:48.054668 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9ba21-0083-48d4-9dc8-46393ce5e605 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:48.061505 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15eb48c-0ae9-4495-ae13-e7766c951593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:48.062281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eb48c-0ae9-4495-ae13-e7766c951593 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:48.062566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eb48c-0ae9-4495-ae13-e7766c951593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:48.062837 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eb48c-0ae9-4495-ae13-e7766c951593 admin admin] PATH_INFO: `/v3/projects/8c6c243829fe4865b0513fdaec55d0b8/users/701782078ed44f0fb89be2f58edac4db/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:48.065554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15eb48c-0ae9-4495-ae13-e7766c951593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c6c243829fe4865b0513fdaec55d0b8, user_id=701782078ed44f0fb89be2f58edac4db, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:48.068829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15eb48c-0ae9-4495-ae13-e7766c951593 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:48.085646 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-c316b1a9-8bf6-4806-a06f-455010a17b94 None None] MFA Rules not processed for user `6e9209e77ce44b128e25a8ddcf78f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:48.091132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c316b1a9-8bf6-4806-a06f-455010a17b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:48.091707 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 816/1636] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:48 2022] PUT /identity/v3/projects/8c6c243829fe4865b0513fdaec55d0b8/users/701782078ed44f0fb89be2f58edac4db/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:48.093538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15eb48c-0ae9-4495-ae13-e7766c951593 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:48.149551 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:48.215064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15eb48c-0ae9-4495-ae13-e7766c951593 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:48.217860 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 821/1637] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Sep 8 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:48.222351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef5af7-311d-4f11-84f5-2663e398649d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:48.223751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef5af7-311d-4f11-84f5-2663e398649d admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:48.224031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef5af7-311d-4f11-84f5-2663e398649d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:48.224304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef5af7-311d-4f11-84f5-2663e398649d admin admin] PATH_INFO: `/v3/projects/45a000f6a7cf4acf80da045a99630f91/users/eb84d447806249a48f85a563c3183aee/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:48.227281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef5af7-311d-4f11-84f5-2663e398649d admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a000f6a7cf4acf80da045a99630f91, user_id=eb84d447806249a48f85a563c3183aee, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:48.230420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef5af7-311d-4f11-84f5-2663e398649d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:48.235765 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40597c-d792-42f7-b2bd-b61a1ee33019 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:48.236190 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40597c-d792-42f7-b2bd-b61a1ee33019 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:48.236512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40597c-d792-42f7-b2bd-b61a1ee33019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:48.246872 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 817/1638] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:48 2022] PUT /identity/v3/projects/45a000f6a7cf4acf80da045a99630f91/users/eb84d447806249a48f85a563c3183aee/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:48.248053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef5af7-311d-4f11-84f5-2663e398649d admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:48.282819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-ce40597c-d792-42f7-b2bd-b61a1ee33019 None None] MFA Rules not processed for user `6e9209e77ce44b128e25a8ddcf78f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:48.288169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ce40597c-d792-42f7-b2bd-b61a1ee33019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:48.292093 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:48.343582 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef5af7-311d-4f11-84f5-2663e398649d admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:48.348735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ddd34-eaeb-43a6-b986-3e4374f253e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:48.350546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ddd34-eaeb-43a6-b986-3e4374f253e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:48.352349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ddd34-eaeb-43a6-b986-3e4374f253e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:48.352660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ddd34-eaeb-43a6-b986-3e4374f253e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:48.391180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 822/1639] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Sep 8 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:48.421213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ef76a-3a8f-4871-9001-ad864c0e22da None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:48.421574 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ef76a-3a8f-4871-9001-ad864c0e22da None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:48.421927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ef76a-3a8f-4871-9001-ad864c0e22da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:48.485554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ddd34-eaeb-43a6-b986-3e4374f253e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:48.487031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ddd34-eaeb-43a6-b986-3e4374f253e5 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:48.515809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 818/1640] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:07:48.517449 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ddd34-eaeb-43a6-b986-3e4374f253e5 service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:48.531281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-003ef76a-3a8f-4871-9001-ad864c0e22da None None] MFA Rules not processed for user `eb84d447806249a48f85a563c3183aee`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:48.535980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-003ef76a-3a8f-4871-9001-ad864c0e22da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:48.539801 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ddd34-eaeb-43a6-b986-3e4374f253e5 service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:48.553683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a8554-47dc-4768-8d33-55e22f72295f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:48.554335 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a8554-47dc-4768-8d33-55e22f72295f service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:48.554604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a8554-47dc-4768-8d33-55e22f72295f service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:48.554875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a8554-47dc-4768-8d33-55e22f72295f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:48.584197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a8554-47dc-4768-8d33-55e22f72295f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:48.593960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680a8554-47dc-4768-8d33-55e22f72295f service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:48.628585 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 819/1641] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:07:48.699243 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 823/1642] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:48.725497 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a8554-47dc-4768-8d33-55e22f72295f service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:48.742963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a8554-47dc-4768-8d33-55e22f72295f service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:48.751729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73c535-97b5-423b-9319-70348e23a845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:48.752664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73c535-97b5-423b-9319-70348e23a845 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:48.753233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73c535-97b5-423b-9319-70348e23a845 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:48.753886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73c535-97b5-423b-9319-70348e23a845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:48.772346 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73c535-97b5-423b-9319-70348e23a845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:48.774925 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73c535-97b5-423b-9319-70348e23a845 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:48.788038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 820/1643] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:48 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:07:49.009867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ef76a-3a8f-4871-9001-ad864c0e22da None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:49.016804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73c535-97b5-423b-9319-70348e23a845 service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:49.025661 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73c535-97b5-423b-9319-70348e23a845 service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:49.031133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7f952-dceb-439f-93c3-8b8b00602efc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:49.031831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7f952-dceb-439f-93c3-8b8b00602efc service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:49.032110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7f952-dceb-439f-93c3-8b8b00602efc service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:49.032386 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7f952-dceb-439f-93c3-8b8b00602efc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:49.119536 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:49.147760 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7f952-dceb-439f-93c3-8b8b00602efc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:49.149227 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc7f952-dceb-439f-93c3-8b8b00602efc service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:49.176463 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ef76a-3a8f-4871-9001-ad864c0e22da None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:49.191722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1646fcf-4906-4800-96a1-9e9cbf6fffc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:49.192378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 821/1644] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:49 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:07:49.193471 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1646fcf-4906-4800-96a1-9e9cbf6fffc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:49.193958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1646fcf-4906-4800-96a1-9e9cbf6fffc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:49.194237 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1646fcf-4906-4800-96a1-9e9cbf6fffc3 admin admin] PATH_INFO: `/v3/users/2ffcc5ea76744215b8a4a7ea49982913` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:49.195727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1646fcf-4906-4800-96a1-9e9cbf6fffc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ffcc5ea76744215b8a4a7ea49982913)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:49.197308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: WARNING py.warnings [None req-a1646fcf-4906-4800-96a1-9e9cbf6fffc3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 16:07:49.197308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: warnings.warn(msg) Sep 08 16:07:49.197308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: Sep 08 16:07:49.198948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1646fcf-4906-4800-96a1-9e9cbf6fffc3 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:49.373521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a1646fcf-4906-4800-96a1-9e9cbf6fffc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ffcc5ea76744215b8a4a7ea49982913'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:49.390165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a1646fcf-4906-4800-96a1-9e9cbf6fffc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ffcc5ea76744215b8a4a7ea49982913'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:49.391107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a1646fcf-4906-4800-96a1-9e9cbf6fffc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ffcc5ea76744215b8a4a7ea49982913'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:49.392121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a1646fcf-4906-4800-96a1-9e9cbf6fffc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ffcc5ea76744215b8a4a7ea49982913'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:49.461334 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a1646fcf-4906-4800-96a1-9e9cbf6fffc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ffcc5ea76744215b8a4a7ea49982913'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:49.535466 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 824/1645] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:07:49 2022] DELETE /identity/v3/users/2ffcc5ea76744215b8a4a7ea49982913 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:49.540930 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b821e8d-2e7a-4555-828c-d8e82e81d705 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:49.541510 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b821e8d-2e7a-4555-828c-d8e82e81d705 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:49.541510 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b821e8d-2e7a-4555-828c-d8e82e81d705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:49.594710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-0b821e8d-2e7a-4555-828c-d8e82e81d705 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:49.600423 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0b821e8d-2e7a-4555-828c-d8e82e81d705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:49.641059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a6a76-b6bc-49ac-b954-a9d5537b4b94 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:49.641352 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a6a76-b6bc-49ac-b954-a9d5537b4b94 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:49.641684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a6a76-b6bc-49ac-b954-a9d5537b4b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:49.684322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-1f8a6a76-b6bc-49ac-b954-a9d5537b4b94 None None] MFA Rules not processed for user `701782078ed44f0fb89be2f58edac4db`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:49.685558 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-1f8a6a76-b6bc-49ac-b954-a9d5537b4b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:49.691611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:49.812104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 822/1646] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:07:49.822910 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b821e8d-2e7a-4555-828c-d8e82e81d705 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:49.825311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0b821e8d-2e7a-4555-828c-d8e82e81d705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:49.835817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 825/1647] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:49 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:07:49.912891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b821e8d-2e7a-4555-828c-d8e82e81d705 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:49.917487 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f8f4a-6b0b-4b64-8336-505558c59592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:49.919591 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f8f4a-6b0b-4b64-8336-505558c59592 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:49.920139 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f8f4a-6b0b-4b64-8336-505558c59592 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:49.920422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f8f4a-6b0b-4b64-8336-505558c59592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:49.934488 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f8f4a-6b0b-4b64-8336-505558c59592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:49.936729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f8f4a-6b0b-4b64-8336-505558c59592 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:49.951965 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 823/1648] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:07:50.968690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf414a4b-4a5c-45b9-ab52-b556a7bbb33e None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:50.969351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf414a4b-4a5c-45b9-ab52-b556a7bbb33e None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:50.969891 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf414a4b-4a5c-45b9-ab52-b556a7bbb33e None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:50.971743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 826/1649] 10.222.0.27 () {54 vars in 921 bytes} [Thu Sep 8 16:07:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:07:50.980737 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9758b-a036-4f6a-bcc5-9672b9f1ee9d None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:50.981921 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9758b-a036-4f6a-bcc5-9672b9f1ee9d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:50.982503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c9758b-a036-4f6a-bcc5-9672b9f1ee9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:51.065791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-32c9758b-a036-4f6a-bcc5-9672b9f1ee9d None None] MFA Rules not processed for user `dd28bb31b55e45668212c9ab69ed3ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:51.067211 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-32c9758b-a036-4f6a-bcc5-9672b9f1ee9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:51.287968 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 824/1650] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Sep 8 16:07:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:51.315088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf414a4b-4a5c-45b9-ab52-b556a7bbb33e None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:51.317192 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-cf414a4b-4a5c-45b9-ab52-b556a7bbb33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:51.358925 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf414a4b-4a5c-45b9-ab52-b556a7bbb33e None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:51.378711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a52865-ab77-4ec9-9d0b-78afbced49ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd28bb31b55e45668212c9ab69ed3ac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:51.382357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a52865-ab77-4ec9-9d0b-78afbced49ad service placement] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:51.382661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a52865-ab77-4ec9-9d0b-78afbced49ad service placement] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:51.382932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a52865-ab77-4ec9-9d0b-78afbced49ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:51.464192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9758b-a036-4f6a-bcc5-9672b9f1ee9d None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:51.467152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-32c9758b-a036-4f6a-bcc5-9672b9f1ee9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:51.482712 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a52865-ab77-4ec9-9d0b-78afbced49ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:51.491134 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a52865-ab77-4ec9-9d0b-78afbced49ad service placement] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:51.504632 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c9758b-a036-4f6a-bcc5-9672b9f1ee9d None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:51.521871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b60a2-410f-4bcb-83ea-7500593ffed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:51.521871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b60a2-410f-4bcb-83ea-7500593ffed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:51.521871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b60a2-410f-4bcb-83ea-7500593ffed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:51.521871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b60a2-410f-4bcb-83ea-7500593ffed2 admin admin] PATH_INFO: `/v3/projects/e604a196b20448caa9a0f2196b7f308e` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:51.523763 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b60a2-410f-4bcb-83ea-7500593ffed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e604a196b20448caa9a0f2196b7f308e)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:51.525077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: WARNING py.warnings [None req-444b60a2-410f-4bcb-83ea-7500593ffed2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 16:07:51.525077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: warnings.warn(msg) Sep 08 16:07:51.525077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: Sep 08 16:07:51.526285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b60a2-410f-4bcb-83ea-7500593ffed2 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:51.536739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 827/1651] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:07:51.673052 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-444b60a2-410f-4bcb-83ea-7500593ffed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e604a196b20448caa9a0f2196b7f308e', 'request_id': 'req-444b60a2-410f-4bcb-83ea-7500593ffed2'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:51.684410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-444b60a2-410f-4bcb-83ea-7500593ffed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e604a196b20448caa9a0f2196b7f308e', 'request_id': 'req-444b60a2-410f-4bcb-83ea-7500593ffed2'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:51.707687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-444b60a2-410f-4bcb-83ea-7500593ffed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e604a196b20448caa9a0f2196b7f308e', 'request_id': 'req-444b60a2-410f-4bcb-83ea-7500593ffed2'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:51.756388 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:51.762561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-444b60a2-410f-4bcb-83ea-7500593ffed2 admin admin] The token cache is being invalidate because project e604a196b20448caa9a0f2196b7f308e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:07:51.763761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-444b60a2-410f-4bcb-83ea-7500593ffed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:51.769634 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-444b60a2-410f-4bcb-83ea-7500593ffed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:51.775229 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 825/1652] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:07:51 2022] DELETE /identity/v3/projects/e604a196b20448caa9a0f2196b7f308e => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:07:51.777028 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e3a3e-45b9-45ec-ab36-15fd902f8276 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:51.777319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e3a3e-45b9-45ec-ab36-15fd902f8276 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:51.777770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864e3a3e-45b9-45ec-ab36-15fd902f8276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:51.813410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-864e3a3e-45b9-45ec-ab36-15fd902f8276 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:51.814843 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-864e3a3e-45b9-45ec-ab36-15fd902f8276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:51.902294 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:51.963935 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 828/1653] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:51.965845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36720f95-39b2-4a64-b7e2-dce1353fd11f None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:51.966134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36720f95-39b2-4a64-b7e2-dce1353fd11f None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:51.966407 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36720f95-39b2-4a64-b7e2-dce1353fd11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:52.004726 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-36720f95-39b2-4a64-b7e2-dce1353fd11f None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:52.006003 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-36720f95-39b2-4a64-b7e2-dce1353fd11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:52.024982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 826/1654] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:52.030548 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e3a3e-45b9-45ec-ab36-15fd902f8276 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:52.048055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864e3a3e-45b9-45ec-ab36-15fd902f8276 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:52.054693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b77f9-6551-40c9-8f18-11c2efa4d199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:52.055589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b77f9-6551-40c9-8f18-11c2efa4d199 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:52.056034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b77f9-6551-40c9-8f18-11c2efa4d199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:52.056718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8b77f9-6551-40c9-8f18-11c2efa4d199 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:52.059102 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b77f9-6551-40c9-8f18-11c2efa4d199 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:07:52.059962 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b77f9-6551-40c9-8f18-11c2efa4d199 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:52.062512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8b77f9-6551-40c9-8f18-11c2efa4d199 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:52.084209 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36720f95-39b2-4a64-b7e2-dce1353fd11f None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:52.084406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 829/1655] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:07:52.094203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36720f95-39b2-4a64-b7e2-dce1353fd11f None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:52.099390 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46369a99-8b2f-4626-9543-61964c291315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:52.100470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46369a99-8b2f-4626-9543-61964c291315 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:52.100951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46369a99-8b2f-4626-9543-61964c291315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:52.101419 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46369a99-8b2f-4626-9543-61964c291315 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:52.102313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46369a99-8b2f-4626-9543-61964c291315 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:52.104479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46369a99-8b2f-4626-9543-61964c291315 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:52.151182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 827/1656] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:07:52 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:07:52.152947 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b77f9-6551-40c9-8f18-11c2efa4d199 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:52.170324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8b77f9-6551-40c9-8f18-11c2efa4d199 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:52.241875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46369a99-8b2f-4626-9543-61964c291315 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:52.244730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-46369a99-8b2f-4626-9543-61964c291315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:52.342228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3627e3-ade3-4a97-9586-0f011527d46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:52.342858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3627e3-ade3-4a97-9586-0f011527d46f admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:52.343930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3627e3-ade3-4a97-9586-0f011527d46f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:52.344121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3627e3-ade3-4a97-9586-0f011527d46f admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:52.344800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3627e3-ade3-4a97-9586-0f011527d46f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:52.348179 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3627e3-ade3-4a97-9586-0f011527d46f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:52.407960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46369a99-8b2f-4626-9543-61964c291315 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:52.409013 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 830/1657] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:07:52 2022] POST /identity/v3/users => generated 287 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:52.409968 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3627e3-ade3-4a97-9586-0f011527d46f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:52.420549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d9484-4495-4d41-8c63-4dee38d93fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:52.421226 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d9484-4495-4d41-8c63-4dee38d93fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:52.421561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d9484-4495-4d41-8c63-4dee38d93fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:52.421838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60d9484-4495-4d41-8c63-4dee38d93fa4 admin admin] PATH_INFO: `/v3/users/911d0d25310b4684af29485659a2578f` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:52.426607 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d9484-4495-4d41-8c63-4dee38d93fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=911d0d25310b4684af29485659a2578f)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:52.428806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60d9484-4495-4d41-8c63-4dee38d93fa4 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:52.429820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3627e3-ade3-4a97-9586-0f011527d46f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:52.444268 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879ad57-5eae-40fc-a25f-acedfe24fc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:52.444912 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879ad57-5eae-40fc-a25f-acedfe24fc8f admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:52.445190 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879ad57-5eae-40fc-a25f-acedfe24fc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:52.445458 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879ad57-5eae-40fc-a25f-acedfe24fc8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:52.446187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879ad57-5eae-40fc-a25f-acedfe24fc8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:52.450881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879ad57-5eae-40fc-a25f-acedfe24fc8f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:52.482365 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 831/1658] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:52.489638 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879ad57-5eae-40fc-a25f-acedfe24fc8f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:52.535280 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879ad57-5eae-40fc-a25f-acedfe24fc8f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:52.540215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a60d9484-4495-4d41-8c63-4dee38d93fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '911d0d25310b4684af29485659a2578f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:52.559223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cf1f7-e932-409b-b2b5-fe196a7d700d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:52.560110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a60d9484-4495-4d41-8c63-4dee38d93fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '911d0d25310b4684af29485659a2578f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:52.560813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a60d9484-4495-4d41-8c63-4dee38d93fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '911d0d25310b4684af29485659a2578f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:52.561143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cf1f7-e932-409b-b2b5-fe196a7d700d admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:52.561355 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cf1f7-e932-409b-b2b5-fe196a7d700d admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:52.561514 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a60d9484-4495-4d41-8c63-4dee38d93fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '911d0d25310b4684af29485659a2578f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:52.561599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cf1f7-e932-409b-b2b5-fe196a7d700d admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:52.562334 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cf1f7-e932-409b-b2b5-fe196a7d700d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:52.567074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0cf1f7-e932-409b-b2b5-fe196a7d700d admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:52.574129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a60d9484-4495-4d41-8c63-4dee38d93fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '911d0d25310b4684af29485659a2578f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:52.611710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 828/1659] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:07:52 2022] DELETE /identity/v3/users/911d0d25310b4684af29485659a2578f => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:52.611710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d9484-4495-4d41-8c63-4dee38d93fa4 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:52.611918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 832/1660] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:52.616309 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a60d9484-4495-4d41-8c63-4dee38d93fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:52.729499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:52.768779 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60d9484-4495-4d41-8c63-4dee38d93fa4 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:52.782062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c6a86-8f71-411e-83af-a46edccd7975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:52.783049 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c6a86-8f71-411e-83af-a46edccd7975 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:52.783484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c6a86-8f71-411e-83af-a46edccd7975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:52.783927 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6c6a86-8f71-411e-83af-a46edccd7975 admin admin] PATH_INFO: `/v3/projects/393b318e788f46fbbfcc1cc91ae4a77a/users/1425ba29cbc94a15b2e273a2e34fbce1/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:52.818788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c6a86-8f71-411e-83af-a46edccd7975 admin admin] RBAC: Authorizing `identity:create_grant(project_id=393b318e788f46fbbfcc1cc91ae4a77a, user_id=1425ba29cbc94a15b2e273a2e34fbce1, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:52.823632 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6c6a86-8f71-411e-83af-a46edccd7975 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:52.850554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 829/1661] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:52 2022] PUT /identity/v3/projects/393b318e788f46fbbfcc1cc91ae4a77a/users/1425ba29cbc94a15b2e273a2e34fbce1/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:52.863925 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c76e7-5911-4261-b0cd-cbf8e4fa36cf None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:52.864205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c76e7-5911-4261-b0cd-cbf8e4fa36cf None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:52.864478 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c76e7-5911-4261-b0cd-cbf8e4fa36cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:52.922013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c6a86-8f71-411e-83af-a46edccd7975 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:52.923592 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9f6c6a86-8f71-411e-83af-a46edccd7975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:52.952859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-d26c76e7-5911-4261-b0cd-cbf8e4fa36cf None None] MFA Rules not processed for user `1425ba29cbc94a15b2e273a2e34fbce1`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:52.954255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d26c76e7-5911-4261-b0cd-cbf8e4fa36cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:53.044946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 833/1662] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:53.045904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c76e7-5911-4261-b0cd-cbf8e4fa36cf None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:53.082212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6c6a86-8f71-411e-83af-a46edccd7975 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:53.088942 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:53.089868 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:53.090231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:53.090565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b admin admin] PATH_INFO: `/v3/projects/ef161043b44b40999d6b227b58a369b1` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:53.091888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef161043b44b40999d6b227b58a369b1)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:53.094308 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:53.098418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:53.171875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef161043b44b40999d6b227b58a369b1', 'request_id': 'req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:53.187617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c76e7-5911-4261-b0cd-cbf8e4fa36cf None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:53.187888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef161043b44b40999d6b227b58a369b1', 'request_id': 'req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:53.197586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef161043b44b40999d6b227b58a369b1', 'request_id': 'req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:53.216330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b admin admin] The token cache is being invalidate because project ef161043b44b40999d6b227b58a369b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:07:53.217159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:53.219636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:53.221944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 830/1663] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:07:52 2022] DELETE /identity/v3/projects/ef161043b44b40999d6b227b58a369b1 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:53.230130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:53.232924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:53.274342 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acd00b-e721-40cf-8a30-c8b48c399ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:53.274946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd00b-e721-40cf-8a30-c8b48c399ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:53.275418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd00b-e721-40cf-8a30-c8b48c399ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:53.275761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4acd00b-e721-40cf-8a30-c8b48c399ca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:53.278392 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4acd00b-e721-40cf-8a30-c8b48c399ca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:53.280717 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4acd00b-e721-40cf-8a30-c8b48c399ca7 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:53.328904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 834/1664] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:07:53 2022] POST /identity/v3/projects => generated 373 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:53.334032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acd00b-e721-40cf-8a30-c8b48c399ca7 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:53.337084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d4acd00b-e721-40cf-8a30-c8b48c399ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:53.376772 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bc0334-2f1f-4d5d-9b8a-426150ec592b admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:53.382909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:53.495668 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4acd00b-e721-40cf-8a30-c8b48c399ca7 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:53.509338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5f598-f712-4327-a8d5-b42891390510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:53.510181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5f598-f712-4327-a8d5-b42891390510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:53.510828 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5f598-f712-4327-a8d5-b42891390510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:53.511268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5f598-f712-4327-a8d5-b42891390510 admin admin] PATH_INFO: `/v3/users/701782078ed44f0fb89be2f58edac4db` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:53.512366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b88117-5f53-44fc-92a5-c6ed2118647e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:53.512886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5f598-f712-4327-a8d5-b42891390510 admin admin] RBAC: Authorizing `identity:delete_user(user_id=701782078ed44f0fb89be2f58edac4db)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:53.513001 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b88117-5f53-44fc-92a5-c6ed2118647e admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:53.513278 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b88117-5f53-44fc-92a5-c6ed2118647e admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:53.513589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b88117-5f53-44fc-92a5-c6ed2118647e admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:53.514245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b88117-5f53-44fc-92a5-c6ed2118647e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:53.517277 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b88117-5f53-44fc-92a5-c6ed2118647e admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:53.521621 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5f598-f712-4327-a8d5-b42891390510 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:53.582991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 835/1665] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:07:53 2022] POST /identity/v3/users => generated 293 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:53.584151 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b88117-5f53-44fc-92a5-c6ed2118647e admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:53.609297 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b88117-5f53-44fc-92a5-c6ed2118647e admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:53.621186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2674d-63bb-498f-a275-0fab6a80fcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:53.621860 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2674d-63bb-498f-a275-0fab6a80fcc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:53.622138 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2674d-63bb-498f-a275-0fab6a80fcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:53.622406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2674d-63bb-498f-a275-0fab6a80fcc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:53.623071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2674d-63bb-498f-a275-0fab6a80fcc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:53.624762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2674d-63bb-498f-a275-0fab6a80fcc2 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:53.657845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 836/1666] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:53.659570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2674d-63bb-498f-a275-0fab6a80fcc2 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:53.676304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f6d5f598-f712-4327-a8d5-b42891390510 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '701782078ed44f0fb89be2f58edac4db'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:53.693929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2674d-63bb-498f-a275-0fab6a80fcc2 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:53.694715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f6d5f598-f712-4327-a8d5-b42891390510 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '701782078ed44f0fb89be2f58edac4db'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:53.695685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f6d5f598-f712-4327-a8d5-b42891390510 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '701782078ed44f0fb89be2f58edac4db'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:53.696339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f6d5f598-f712-4327-a8d5-b42891390510 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '701782078ed44f0fb89be2f58edac4db'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:53.707501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef54bd-837c-4eb5-b500-3186486bdc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:53.708194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef54bd-837c-4eb5-b500-3186486bdc26 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:53.708478 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef54bd-837c-4eb5-b500-3186486bdc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:53.708749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef54bd-837c-4eb5-b500-3186486bdc26 admin admin] PATH_INFO: `/v3/projects/5bec3f2d6bf542158802cfe561d9394f/users/3bd6a7430aef4026a24a7980b9a11e6f/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:53.710417 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f6d5f598-f712-4327-a8d5-b42891390510 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '701782078ed44f0fb89be2f58edac4db'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:53.755573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef54bd-837c-4eb5-b500-3186486bdc26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bec3f2d6bf542158802cfe561d9394f, user_id=3bd6a7430aef4026a24a7980b9a11e6f, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:53.758565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef54bd-837c-4eb5-b500-3186486bdc26 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:53.770059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 831/1667] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:07:53 2022] DELETE /identity/v3/users/701782078ed44f0fb89be2f58edac4db => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:07:53.799110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 837/1668] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:53 2022] PUT /identity/v3/projects/5bec3f2d6bf542158802cfe561d9394f/users/3bd6a7430aef4026a24a7980b9a11e6f/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:53.805406 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5f598-f712-4327-a8d5-b42891390510 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:53.806953 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f6d5f598-f712-4327-a8d5-b42891390510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:53.981729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5f598-f712-4327-a8d5-b42891390510 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:54.046195 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:54.110882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59850a-df64-41b6-8f88-3353d65f369d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:54.112575 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59850a-df64-41b6-8f88-3353d65f369d admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:54.113052 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59850a-df64-41b6-8f88-3353d65f369d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:54.113653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59850a-df64-41b6-8f88-3353d65f369d admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:54.114518 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59850a-df64-41b6-8f88-3353d65f369d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:54.116672 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59850a-df64-41b6-8f88-3353d65f369d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:54.140855 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 832/1669] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:54.142576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef54bd-837c-4eb5-b500-3186486bdc26 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:54.154390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:54.164834 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59850a-df64-41b6-8f88-3353d65f369d admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:54.166802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef54bd-837c-4eb5-b500-3186486bdc26 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:54.172173 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ad59850a-df64-41b6-8f88-3353d65f369d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:54.185795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b009b-dfaa-4077-a76b-eb559bd81fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:54.186959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b009b-dfaa-4077-a76b-eb559bd81fed admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:54.186959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b009b-dfaa-4077-a76b-eb559bd81fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:54.190799 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b009b-dfaa-4077-a76b-eb559bd81fed admin admin] PATH_INFO: `/v3/projects/5bec3f2d6bf542158802cfe561d9394f/users/3bd6a7430aef4026a24a7980b9a11e6f/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:54.193894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b009b-dfaa-4077-a76b-eb559bd81fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bec3f2d6bf542158802cfe561d9394f, user_id=3bd6a7430aef4026a24a7980b9a11e6f, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:54.197742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b009b-dfaa-4077-a76b-eb559bd81fed admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:54.203949 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59850a-df64-41b6-8f88-3353d65f369d admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:54.211182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b7cfe6-3b41-4217-b7df-b49da12f10cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:54.211938 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7cfe6-3b41-4217-b7df-b49da12f10cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:54.212215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7cfe6-3b41-4217-b7df-b49da12f10cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:54.212938 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b7cfe6-3b41-4217-b7df-b49da12f10cf admin admin] PATH_INFO: `/v3/projects/8c6c243829fe4865b0513fdaec55d0b8` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:54.213664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7cfe6-3b41-4217-b7df-b49da12f10cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c6c243829fe4865b0513fdaec55d0b8)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:54.218093 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b7cfe6-3b41-4217-b7df-b49da12f10cf admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:54.232004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 838/1670] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:54 2022] PUT /identity/v3/projects/5bec3f2d6bf542158802cfe561d9394f/users/3bd6a7430aef4026a24a7980b9a11e6f/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:54.242608 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b3e8e-edd4-4195-9bb2-a8b21c7ba29c None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:54.243133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b3e8e-edd4-4195-9bb2-a8b21c7ba29c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:54.243598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b3e8e-edd4-4195-9bb2-a8b21c7ba29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:54.283070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-29b7cfe6-3b41-4217-b7df-b49da12f10cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c6c243829fe4865b0513fdaec55d0b8', 'request_id': 'req-29b7cfe6-3b41-4217-b7df-b49da12f10cf'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:54.294455 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-29b7cfe6-3b41-4217-b7df-b49da12f10cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c6c243829fe4865b0513fdaec55d0b8', 'request_id': 'req-29b7cfe6-3b41-4217-b7df-b49da12f10cf'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:54.302201 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-29b7cfe6-3b41-4217-b7df-b49da12f10cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c6c243829fe4865b0513fdaec55d0b8', 'request_id': 'req-29b7cfe6-3b41-4217-b7df-b49da12f10cf'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:54.307068 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-d41b3e8e-edd4-4195-9bb2-a8b21c7ba29c None None] MFA Rules not processed for user `3bd6a7430aef4026a24a7980b9a11e6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:54.309057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d41b3e8e-edd4-4195-9bb2-a8b21c7ba29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:54.320763 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-29b7cfe6-3b41-4217-b7df-b49da12f10cf admin admin] The token cache is being invalidate because project 8c6c243829fe4865b0513fdaec55d0b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:07:54.321220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-29b7cfe6-3b41-4217-b7df-b49da12f10cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:54.323183 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-29b7cfe6-3b41-4217-b7df-b49da12f10cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:54.325297 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 833/1671] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:07:54 2022] DELETE /identity/v3/projects/8c6c243829fe4865b0513fdaec55d0b8 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:54.330406 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a0cca-f8a9-48c1-98ba-5cca52a36c89 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:54.330868 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a0cca-f8a9-48c1-98ba-5cca52a36c89 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:54.331730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a0cca-f8a9-48c1-98ba-5cca52a36c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:54.380336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-c56a0cca-f8a9-48c1-98ba-5cca52a36c89 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:54.382537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c56a0cca-f8a9-48c1-98ba-5cca52a36c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:54.405454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 839/1672] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:07:54.419166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b3e8e-edd4-4195-9bb2-a8b21c7ba29c None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:54.421334 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d41b3e8e-edd4-4195-9bb2-a8b21c7ba29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:54.465952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 834/1673] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:54.476029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1ebd8-6e8b-4230-8ad6-274366157576 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:54.476475 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1ebd8-6e8b-4230-8ad6-274366157576 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:54.476852 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1ebd8-6e8b-4230-8ad6-274366157576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:54.504324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b3e8e-edd4-4195-9bb2-a8b21c7ba29c None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:54.510225 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9056a6-8688-43ee-b0b0-8135dc465ea4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:54.511057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9056a6-8688-43ee-b0b0-8135dc465ea4 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:54.511392 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9056a6-8688-43ee-b0b0-8135dc465ea4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:54.511834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9056a6-8688-43ee-b0b0-8135dc465ea4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:54.515187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-e6c1ebd8-6e8b-4230-8ad6-274366157576 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:54.516770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e6c1ebd8-6e8b-4230-8ad6-274366157576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:54.526522 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 835/1674] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:54.533354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1ebd8-6e8b-4230-8ad6-274366157576 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:54.534167 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9056a6-8688-43ee-b0b0-8135dc465ea4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:54.536145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9056a6-8688-43ee-b0b0-8135dc465ea4 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:54.561094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1ebd8-6e8b-4230-8ad6-274366157576 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:54.565396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 840/1675] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:07:54.573107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79274987-563f-4716-a9d2-ae4cd45eef04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:54.573696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79274987-563f-4716-a9d2-ae4cd45eef04 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:54.573974 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79274987-563f-4716-a9d2-ae4cd45eef04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:54.574478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79274987-563f-4716-a9d2-ae4cd45eef04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:54.575049 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79274987-563f-4716-a9d2-ae4cd45eef04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:07:54.575353 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79274987-563f-4716-a9d2-ae4cd45eef04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:54.585098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79274987-563f-4716-a9d2-ae4cd45eef04 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:54.613553 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 836/1676] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:07:54.615380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9056a6-8688-43ee-b0b0-8135dc465ea4 service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:54.629853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9056a6-8688-43ee-b0b0-8135dc465ea4 service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:54.637643 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cd788-83c9-47aa-a0ea-4b246b6d68bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:54.640647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cd788-83c9-47aa-a0ea-4b246b6d68bf admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:54.641145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cd788-83c9-47aa-a0ea-4b246b6d68bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:54.641475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cd788-83c9-47aa-a0ea-4b246b6d68bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:54.642174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cd788-83c9-47aa-a0ea-4b246b6d68bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:54.645723 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cd788-83c9-47aa-a0ea-4b246b6d68bf admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:54.717210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 841/1677] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:07:54 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:54.718594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79274987-563f-4716-a9d2-ae4cd45eef04 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:54.748673 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79274987-563f-4716-a9d2-ae4cd45eef04 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:54.846387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:54.960068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70d979-117e-4e0a-9e74-ceeb6795b3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:54.961447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70d979-117e-4e0a-9e74-ceeb6795b3ee admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:54.961850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70d979-117e-4e0a-9e74-ceeb6795b3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:54.962323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70d979-117e-4e0a-9e74-ceeb6795b3ee admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:54.963088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb70d979-117e-4e0a-9e74-ceeb6795b3ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:54.966231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb70d979-117e-4e0a-9e74-ceeb6795b3ee admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:55.031671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 837/1678] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:07:54 2022] POST /identity/v3/users => generated 290 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:55.035691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cd788-83c9-47aa-a0ea-4b246b6d68bf admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:55.050521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cd788-83c9-47aa-a0ea-4b246b6d68bf admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:55.060533 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e7198-f4eb-4ff8-b6d8-728c49bc9673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:55.061182 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e7198-f4eb-4ff8-b6d8-728c49bc9673 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:55.065076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e7198-f4eb-4ff8-b6d8-728c49bc9673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:55.065434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e7198-f4eb-4ff8-b6d8-728c49bc9673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:55.066119 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e7198-f4eb-4ff8-b6d8-728c49bc9673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:55.067818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e7198-f4eb-4ff8-b6d8-728c49bc9673 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:55.098894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 842/1679] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:55.099998 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70d979-117e-4e0a-9e74-ceeb6795b3ee admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:55.115264 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70d979-117e-4e0a-9e74-ceeb6795b3ee admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:55.122718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04641203-12a5-4393-bd89-548f73687f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:55.123875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04641203-12a5-4393-bd89-548f73687f1a admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:55.124168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04641203-12a5-4393-bd89-548f73687f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:55.124441 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04641203-12a5-4393-bd89-548f73687f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:55.125107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04641203-12a5-4393-bd89-548f73687f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:55.126767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04641203-12a5-4393-bd89-548f73687f1a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:55.172274 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 838/1680] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:07:55.176737 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e7198-f4eb-4ff8-b6d8-728c49bc9673 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:55.208807 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e7198-f4eb-4ff8-b6d8-728c49bc9673 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:55.222244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51d5a6-d1bf-4f21-98c1-485393188c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:55.223346 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51d5a6-d1bf-4f21-98c1-485393188c5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:55.223795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51d5a6-d1bf-4f21-98c1-485393188c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:55.225530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51d5a6-d1bf-4f21-98c1-485393188c5c admin admin] PATH_INFO: `/v3/projects/836c5986c96e424a8d4b8b0e26c0ddc1/users/a7102fe9b2654a079125597e49846614/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:55.261441 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51d5a6-d1bf-4f21-98c1-485393188c5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=836c5986c96e424a8d4b8b0e26c0ddc1, user_id=a7102fe9b2654a079125597e49846614, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:55.289303 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51d5a6-d1bf-4f21-98c1-485393188c5c admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:55.315708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 843/1681] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:55 2022] PUT /identity/v3/projects/836c5986c96e424a8d4b8b0e26c0ddc1/users/a7102fe9b2654a079125597e49846614/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:55.327174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04641203-12a5-4393-bd89-548f73687f1a admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:55.329119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-04641203-12a5-4393-bd89-548f73687f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:55.339575 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638c490-a2f6-4031-9e0d-331b7ecff678 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:55.339575 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638c490-a2f6-4031-9e0d-331b7ecff678 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:55.345215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638c490-a2f6-4031-9e0d-331b7ecff678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:55.360396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04641203-12a5-4393-bd89-548f73687f1a admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:55.453855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-3638c490-a2f6-4031-9e0d-331b7ecff678 None None] MFA Rules not processed for user `a7102fe9b2654a079125597e49846614`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:55.458367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3638c490-a2f6-4031-9e0d-331b7ecff678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:55.499266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:55.528180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06731cb-1770-41c3-8e17-c28a88f7af3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:55.536990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06731cb-1770-41c3-8e17-c28a88f7af3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:55.536990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06731cb-1770-41c3-8e17-c28a88f7af3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:55.536990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06731cb-1770-41c3-8e17-c28a88f7af3a admin admin] PATH_INFO: `/v3/users/1425ba29cbc94a15b2e273a2e34fbce1` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:55.538874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06731cb-1770-41c3-8e17-c28a88f7af3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1425ba29cbc94a15b2e273a2e34fbce1)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:55.541138 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06731cb-1770-41c3-8e17-c28a88f7af3a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:55.640875 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 844/1682] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:55.651789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638c490-a2f6-4031-9e0d-331b7ecff678 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:55.666783 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638c490-a2f6-4031-9e0d-331b7ecff678 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:55.677938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd040a1-e734-4917-9915-843f3b4cd5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:55.679026 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd040a1-e734-4917-9915-843f3b4cd5bc admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:55.679316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd040a1-e734-4917-9915-843f3b4cd5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:55.679643 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd040a1-e734-4917-9915-843f3b4cd5bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:55.680585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd040a1-e734-4917-9915-843f3b4cd5bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:55.682781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd040a1-e734-4917-9915-843f3b4cd5bc admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:55.705553 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c06731cb-1770-41c3-8e17-c28a88f7af3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1425ba29cbc94a15b2e273a2e34fbce1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:55.718798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c06731cb-1770-41c3-8e17-c28a88f7af3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1425ba29cbc94a15b2e273a2e34fbce1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:55.719820 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c06731cb-1770-41c3-8e17-c28a88f7af3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1425ba29cbc94a15b2e273a2e34fbce1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:55.721005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c06731cb-1770-41c3-8e17-c28a88f7af3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1425ba29cbc94a15b2e273a2e34fbce1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:55.726162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 845/1683] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:07:55 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:55.730185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd040a1-e734-4917-9915-843f3b4cd5bc admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:55.736670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0dd040a1-e734-4917-9915-843f3b4cd5bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:55.747485 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c06731cb-1770-41c3-8e17-c28a88f7af3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1425ba29cbc94a15b2e273a2e34fbce1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:55.786833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 839/1684] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:07:55 2022] DELETE /identity/v3/users/1425ba29cbc94a15b2e273a2e34fbce1 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:55.788722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbf5f7-2b8e-49ab-8834-467b80ed6aba None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:55.789006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbf5f7-2b8e-49ab-8834-467b80ed6aba None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:55.789608 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbf5f7-2b8e-49ab-8834-467b80ed6aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:55.816943 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-debbf5f7-2b8e-49ab-8834-467b80ed6aba None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:55.819657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-debbf5f7-2b8e-49ab-8834-467b80ed6aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:55.888490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:55.890319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd040a1-e734-4917-9915-843f3b4cd5bc admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:55.976235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 840/1685] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:55.991275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbf5f7-2b8e-49ab-8834-467b80ed6aba None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:56.006737 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-debbf5f7-2b8e-49ab-8834-467b80ed6aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:56.068291 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142aab6-917d-40d0-9e83-96c50539e701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:56.068952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142aab6-917d-40d0-9e83-96c50539e701 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:56.069235 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142aab6-917d-40d0-9e83-96c50539e701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:56.069511 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3142aab6-917d-40d0-9e83-96c50539e701 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:56.070193 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142aab6-917d-40d0-9e83-96c50539e701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:56.075689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3142aab6-917d-40d0-9e83-96c50539e701 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:56.136109 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 846/1686] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:07:55 2022] POST /identity/v3/users => generated 296 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:56.144792 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142aab6-917d-40d0-9e83-96c50539e701 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:56.153095 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbf5f7-2b8e-49ab-8834-467b80ed6aba None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:56.164222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeee681-7448-40f9-b561-5b2b4a2ce040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:56.164617 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeee681-7448-40f9-b561-5b2b4a2ce040 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:56.164899 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeee681-7448-40f9-b561-5b2b4a2ce040 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:56.165172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeee681-7448-40f9-b561-5b2b4a2ce040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:56.171675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3142aab6-917d-40d0-9e83-96c50539e701 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:56.194271 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeee681-7448-40f9-b561-5b2b4a2ce040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:56.195433 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15764a-7c59-42e0-8493-4e95d1eb4eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:56.195916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeee681-7448-40f9-b561-5b2b4a2ce040 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:56.196088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15764a-7c59-42e0-8493-4e95d1eb4eff admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:56.196525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15764a-7c59-42e0-8493-4e95d1eb4eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:56.197318 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15764a-7c59-42e0-8493-4e95d1eb4eff admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:56.198132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15764a-7c59-42e0-8493-4e95d1eb4eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:56.200065 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15764a-7c59-42e0-8493-4e95d1eb4eff admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:56.208851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 841/1687] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:07:56.233590 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 847/1688] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:56.235876 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeee681-7448-40f9-b561-5b2b4a2ce040 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:56.258222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeee681-7448-40f9-b561-5b2b4a2ce040 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:56.284468 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b67be-1cc6-4544-8a70-45721657caa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:56.285411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b67be-1cc6-4544-8a70-45721657caa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:56.285845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b67be-1cc6-4544-8a70-45721657caa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:56.286738 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b67be-1cc6-4544-8a70-45721657caa0 admin admin] PATH_INFO: `/v3/projects/884490340a3c402d992fe5fb1aa355a2/users/feb74b3c90ad4904a8dd3b05359908dc/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:56.329882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b67be-1cc6-4544-8a70-45721657caa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=884490340a3c402d992fe5fb1aa355a2, user_id=feb74b3c90ad4904a8dd3b05359908dc, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:56.332025 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0b67be-1cc6-4544-8a70-45721657caa0 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:56.357959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 842/1689] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:56 2022] PUT /identity/v3/projects/884490340a3c402d992fe5fb1aa355a2/users/feb74b3c90ad4904a8dd3b05359908dc/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:07:56.361971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15764a-7c59-42e0-8493-4e95d1eb4eff admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:56.372757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15764a-7c59-42e0-8493-4e95d1eb4eff admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:56.389431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:56.483902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f5c3ec-7bb0-4f9f-9c0b-f0d6cedeb384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:56.484549 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5c3ec-7bb0-4f9f-9c0b-f0d6cedeb384 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:56.484824 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5c3ec-7bb0-4f9f-9c0b-f0d6cedeb384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:56.485092 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f5c3ec-7bb0-4f9f-9c0b-f0d6cedeb384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:56.485744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5c3ec-7bb0-4f9f-9c0b-f0d6cedeb384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:56.489140 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f5c3ec-7bb0-4f9f-9c0b-f0d6cedeb384 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:56.521057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 848/1690] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:56.522088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b67be-1cc6-4544-8a70-45721657caa0 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:56.549280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b67be-1cc6-4544-8a70-45721657caa0 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:56.563806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d8228-990b-4a0f-affd-21c5f1d498f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:56.564507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d8228-990b-4a0f-affd-21c5f1d498f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:56.567345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d8228-990b-4a0f-affd-21c5f1d498f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:56.567431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d8228-990b-4a0f-affd-21c5f1d498f4 admin admin] PATH_INFO: `/v3/projects/884490340a3c402d992fe5fb1aa355a2/users/feb74b3c90ad4904a8dd3b05359908dc/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:56.574662 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d8228-990b-4a0f-affd-21c5f1d498f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=884490340a3c402d992fe5fb1aa355a2, user_id=feb74b3c90ad4904a8dd3b05359908dc, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:56.581111 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d8228-990b-4a0f-affd-21c5f1d498f4 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:56.605058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 843/1691] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:56 2022] PUT /identity/v3/projects/884490340a3c402d992fe5fb1aa355a2/users/feb74b3c90ad4904a8dd3b05359908dc/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:56.617399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755abea-7e49-4220-86a7-cdf41a3aff5a None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:56.617847 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755abea-7e49-4220-86a7-cdf41a3aff5a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:56.618165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c755abea-7e49-4220-86a7-cdf41a3aff5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:56.682873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-c755abea-7e49-4220-86a7-cdf41a3aff5a None None] MFA Rules not processed for user `feb74b3c90ad4904a8dd3b05359908dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:56.687011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c755abea-7e49-4220-86a7-cdf41a3aff5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:56.772061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d8228-990b-4a0f-affd-21c5f1d498f4 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:56.773826 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f32d8228-990b-4a0f-affd-21c5f1d498f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:56.810894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 849/1692] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:07:56.825597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755abea-7e49-4220-86a7-cdf41a3aff5a None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:56.829269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c755abea-7e49-4220-86a7-cdf41a3aff5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:56.901065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d8228-990b-4a0f-affd-21c5f1d498f4 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:56.908831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:56.909646 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:56.909951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:56.910601 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf admin admin] PATH_INFO: `/v3/projects/393b318e788f46fbbfcc1cc91ae4a77a` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:56.911775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=393b318e788f46fbbfcc1cc91ae4a77a)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:56.920482 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:56.977862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c755abea-7e49-4220-86a7-cdf41a3aff5a None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:57.008621 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:57.031973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '393b318e788f46fbbfcc1cc91ae4a77a', 'request_id': 'req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:57.052965 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '393b318e788f46fbbfcc1cc91ae4a77a', 'request_id': 'req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:57.062005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '393b318e788f46fbbfcc1cc91ae4a77a', 'request_id': 'req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:57.089508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf admin admin] The token cache is being invalidate because project 393b318e788f46fbbfcc1cc91ae4a77a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:07:57.089883 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:57.090847 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:57.095497 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 844/1693] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:07:56 2022] DELETE /identity/v3/projects/393b318e788f46fbbfcc1cc91ae4a77a => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:07:57.096588 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:57.098016 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:57.128773 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996a6ac-ab43-4ea7-bb0f-f2c364dbf0d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:57.129604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996a6ac-ab43-4ea7-bb0f-f2c364dbf0d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:57.135333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996a6ac-ab43-4ea7-bb0f-f2c364dbf0d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:57.135854 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996a6ac-ab43-4ea7-bb0f-f2c364dbf0d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:57.140798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b996a6ac-ab43-4ea7-bb0f-f2c364dbf0d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:57.156987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:57.225419 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093f613-2d9e-49bf-bb1f-c2dd1c68dbcf admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:57.231086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2d4b0-09e9-42c8-9534-8fc14818a4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:57.231931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2d4b0-09e9-42c8-9534-8fc14818a4ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:57.232256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2d4b0-09e9-42c8-9534-8fc14818a4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:57.232537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac2d4b0-09e9-42c8-9534-8fc14818a4ec admin admin] PATH_INFO: `/v3/users/3bd6a7430aef4026a24a7980b9a11e6f` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:57.234093 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2d4b0-09e9-42c8-9534-8fc14818a4ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bd6a7430aef4026a24a7980b9a11e6f)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:57.236434 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac2d4b0-09e9-42c8-9534-8fc14818a4ec admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:57.352793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996a6ac-ab43-4ea7-bb0f-f2c364dbf0d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:57.354550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996a6ac-ab43-4ea7-bb0f-f2c364dbf0d6 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:57.370033 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-aac2d4b0-09e9-42c8-9534-8fc14818a4ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bd6a7430aef4026a24a7980b9a11e6f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:57.383921 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 850/1694] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:07:57.395584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-aac2d4b0-09e9-42c8-9534-8fc14818a4ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bd6a7430aef4026a24a7980b9a11e6f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:57.396414 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-aac2d4b0-09e9-42c8-9534-8fc14818a4ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bd6a7430aef4026a24a7980b9a11e6f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:57.397202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-aac2d4b0-09e9-42c8-9534-8fc14818a4ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bd6a7430aef4026a24a7980b9a11e6f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:57.417001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-aac2d4b0-09e9-42c8-9534-8fc14818a4ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bd6a7430aef4026a24a7980b9a11e6f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:57.477668 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 845/1695] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:07:57 2022] DELETE /identity/v3/users/3bd6a7430aef4026a24a7980b9a11e6f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:57.543166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996a6ac-ab43-4ea7-bb0f-f2c364dbf0d6 service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:57.544981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b996a6ac-ab43-4ea7-bb0f-f2c364dbf0d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:57.575435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996a6ac-ab43-4ea7-bb0f-f2c364dbf0d6 service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:57.723006 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250cc43-fce2-414d-b4e3-fce6be14684d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:57.725403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250cc43-fce2-414d-b4e3-fce6be14684d service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:57.726160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250cc43-fce2-414d-b4e3-fce6be14684d service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:57.727361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a250cc43-fce2-414d-b4e3-fce6be14684d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:57.734215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a250cc43-fce2-414d-b4e3-fce6be14684d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:57.902355 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a250cc43-fce2-414d-b4e3-fce6be14684d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:57.903865 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a250cc43-fce2-414d-b4e3-fce6be14684d service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:57.933117 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 851/1696] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:07:58.430570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2d4b0-09e9-42c8-9534-8fc14818a4ec admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:58.438091 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-aac2d4b0-09e9-42c8-9534-8fc14818a4ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:58.557161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:58.578511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac2d4b0-09e9-42c8-9534-8fc14818a4ec admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:58.587096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90242de2-8848-4978-b7a9-d23f81ae4185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:58.588176 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90242de2-8848-4978-b7a9-d23f81ae4185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:58.588688 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90242de2-8848-4978-b7a9-d23f81ae4185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:58.589119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90242de2-8848-4978-b7a9-d23f81ae4185 admin admin] PATH_INFO: `/v3/users/44e6d392d538462a875805138980da85` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:58.590945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90242de2-8848-4978-b7a9-d23f81ae4185 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44e6d392d538462a875805138980da85)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:58.593551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90242de2-8848-4978-b7a9-d23f81ae4185 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:58.603604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250cc43-fce2-414d-b4e3-fce6be14684d service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:58.607406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a250cc43-fce2-414d-b4e3-fce6be14684d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:58.664686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-90242de2-8848-4978-b7a9-d23f81ae4185 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44e6d392d538462a875805138980da85'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:58.666053 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a250cc43-fce2-414d-b4e3-fce6be14684d service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:58.682473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-90242de2-8848-4978-b7a9-d23f81ae4185 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44e6d392d538462a875805138980da85'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:58.683282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-90242de2-8848-4978-b7a9-d23f81ae4185 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44e6d392d538462a875805138980da85'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:58.684220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-90242de2-8848-4978-b7a9-d23f81ae4185 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44e6d392d538462a875805138980da85'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:58.689289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01901ea7-f629-4d28-b7f8-8f43b3ec1cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:58.690792 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01901ea7-f629-4d28-b7f8-8f43b3ec1cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:58.691281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01901ea7-f629-4d28-b7f8-8f43b3ec1cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:58.691794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01901ea7-f629-4d28-b7f8-8f43b3ec1cb5 admin admin] PATH_INFO: `/v3/projects/5bec3f2d6bf542158802cfe561d9394f` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:58.695669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-90242de2-8848-4978-b7a9-d23f81ae4185 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44e6d392d538462a875805138980da85'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:58.701369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01901ea7-f629-4d28-b7f8-8f43b3ec1cb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bec3f2d6bf542158802cfe561d9394f)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:58.707111 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01901ea7-f629-4d28-b7f8-8f43b3ec1cb5 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:58.740481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 846/1697] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:07:58 2022] DELETE /identity/v3/users/44e6d392d538462a875805138980da85 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:58.742362 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917e861-df36-471c-a0dc-883ada8cacd2 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:58.742733 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917e861-df36-471c-a0dc-883ada8cacd2 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:58.743128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1917e861-df36-471c-a0dc-883ada8cacd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:58.768765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-1917e861-df36-471c-a0dc-883ada8cacd2 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:58.770337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1917e861-df36-471c-a0dc-883ada8cacd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:58.873939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 847/1698] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:07:58.882596 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-01901ea7-f629-4d28-b7f8-8f43b3ec1cb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bec3f2d6bf542158802cfe561d9394f', 'request_id': 'req-01901ea7-f629-4d28-b7f8-8f43b3ec1cb5'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:58.887098 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:58.903307 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-01901ea7-f629-4d28-b7f8-8f43b3ec1cb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bec3f2d6bf542158802cfe561d9394f', 'request_id': 'req-01901ea7-f629-4d28-b7f8-8f43b3ec1cb5'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:58.915890 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-01901ea7-f629-4d28-b7f8-8f43b3ec1cb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bec3f2d6bf542158802cfe561d9394f', 'request_id': 'req-01901ea7-f629-4d28-b7f8-8f43b3ec1cb5'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:58.922612 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917e861-df36-471c-a0dc-883ada8cacd2 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:58.924025 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1917e861-df36-471c-a0dc-883ada8cacd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:58.957537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-01901ea7-f629-4d28-b7f8-8f43b3ec1cb5 admin admin] The token cache is being invalidate because project 5bec3f2d6bf542158802cfe561d9394f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:07:58.958020 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-01901ea7-f629-4d28-b7f8-8f43b3ec1cb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:58.961567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-01901ea7-f629-4d28-b7f8-8f43b3ec1cb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:07:58.974239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 852/1699] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:07:58 2022] DELETE /identity/v3/projects/5bec3f2d6bf542158802cfe561d9394f => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:07:58.976887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22dde1-fdc8-4360-86bb-53d5fcce9a62 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:58.977371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22dde1-fdc8-4360-86bb-53d5fcce9a62 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:58.977839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22dde1-fdc8-4360-86bb-53d5fcce9a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:59.041421 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1917e861-df36-471c-a0dc-883ada8cacd2 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:59.059522 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-bd22dde1-fdc8-4360-86bb-53d5fcce9a62 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:59.061777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-bd22dde1-fdc8-4360-86bb-53d5fcce9a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:59.074624 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:07:59.135094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66ebe0-6f6f-4a84-a2a3-f48f89e0f45e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:59.136507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66ebe0-6f6f-4a84-a2a3-f48f89e0f45e service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:59.136957 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66ebe0-6f6f-4a84-a2a3-f48f89e0f45e service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:59.137348 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b66ebe0-6f6f-4a84-a2a3-f48f89e0f45e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:59.139767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-8b66ebe0-6f6f-4a84-a2a3-f48f89e0f45e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:59.242689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 853/1700] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:07:59.254695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd8986-fb86-4739-aaf4-a71d92f57511 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:59.255290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd8986-fb86-4739-aaf4-a71d92f57511 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:59.255476 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd8986-fb86-4739-aaf4-a71d92f57511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:59.299104 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-b3dd8986-fb86-4739-aaf4-a71d92f57511 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:07:59.300926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b3dd8986-fb86-4739-aaf4-a71d92f57511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:07:59.325674 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 854/1701] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:07:59.326780 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66ebe0-6f6f-4a84-a2a3-f48f89e0f45e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:59.327119 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd8986-fb86-4739-aaf4-a71d92f57511 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:59.329494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b66ebe0-6f6f-4a84-a2a3-f48f89e0f45e service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:59.337127 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd8986-fb86-4739-aaf4-a71d92f57511 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:59.342454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b851aa-8f4b-430f-a4f5-b82af55fa94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:59.343301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b851aa-8f4b-430f-a4f5-b82af55fa94e admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:59.343807 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b851aa-8f4b-430f-a4f5-b82af55fa94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:59.344260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b851aa-8f4b-430f-a4f5-b82af55fa94e admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:59.345872 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b851aa-8f4b-430f-a4f5-b82af55fa94e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:07:59.346380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b851aa-8f4b-430f-a4f5-b82af55fa94e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:59.348238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b851aa-8f4b-430f-a4f5-b82af55fa94e admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:59.359996 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 848/1702] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:07:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:07:59.381590 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 855/1703] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:07:59.386994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66ebe0-6f6f-4a84-a2a3-f48f89e0f45e service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:59.407952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b66ebe0-6f6f-4a84-a2a3-f48f89e0f45e service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:59.430141 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460f02b-8ff9-43ca-b379-3c5915601c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:59.433046 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460f02b-8ff9-43ca-b379-3c5915601c7b admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:59.434495 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460f02b-8ff9-43ca-b379-3c5915601c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:59.435806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460f02b-8ff9-43ca-b379-3c5915601c7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:59.437549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460f02b-8ff9-43ca-b379-3c5915601c7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:59.456186 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460f02b-8ff9-43ca-b379-3c5915601c7b admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:59.510298 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 849/1704] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:07:59 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:59.511595 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b851aa-8f4b-430f-a4f5-b82af55fa94e admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:59.522287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b851aa-8f4b-430f-a4f5-b82af55fa94e admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:59.620564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819d94b-30b2-4121-bc40-e30b488ae451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:59.621786 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819d94b-30b2-4121-bc40-e30b488ae451 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:59.622207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819d94b-30b2-4121-bc40-e30b488ae451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:59.622589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819d94b-30b2-4121-bc40-e30b488ae451 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:59.623844 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819d94b-30b2-4121-bc40-e30b488ae451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:59.626302 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819d94b-30b2-4121-bc40-e30b488ae451 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:59.682869 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 856/1705] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:07:59 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:07:59.684610 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460f02b-8ff9-43ca-b379-3c5915601c7b admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:59.701742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460f02b-8ff9-43ca-b379-3c5915601c7b admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:59.715778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998af55a-cf61-4cdd-9e18-0a55755afb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:59.718472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998af55a-cf61-4cdd-9e18-0a55755afb5c admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:59.718761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998af55a-cf61-4cdd-9e18-0a55755afb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:59.719624 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998af55a-cf61-4cdd-9e18-0a55755afb5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:59.721634 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998af55a-cf61-4cdd-9e18-0a55755afb5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:59.725592 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998af55a-cf61-4cdd-9e18-0a55755afb5c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:59.755410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 850/1706] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:59.760143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819d94b-30b2-4121-bc40-e30b488ae451 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:59.787378 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819d94b-30b2-4121-bc40-e30b488ae451 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:59.816729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d86a9-78bf-4ca1-86d7-7fb9e189aa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:59.817448 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d86a9-78bf-4ca1-86d7-7fb9e189aa6d admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:59.817728 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d86a9-78bf-4ca1-86d7-7fb9e189aa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:59.817987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d86a9-78bf-4ca1-86d7-7fb9e189aa6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:07:59.818640 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d86a9-78bf-4ca1-86d7-7fb9e189aa6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:07:59.820844 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d86a9-78bf-4ca1-86d7-7fb9e189aa6d admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:07:59.912526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 857/1707] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:07:59.913896 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998af55a-cf61-4cdd-9e18-0a55755afb5c admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:07:59.930015 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998af55a-cf61-4cdd-9e18-0a55755afb5c admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:07:59.957350 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5e8bc-8d97-4b94-8c80-2226b3e87510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:07:59.958081 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5e8bc-8d97-4b94-8c80-2226b3e87510 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:07:59.958361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5e8bc-8d97-4b94-8c80-2226b3e87510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:07:59.958630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5e8bc-8d97-4b94-8c80-2226b3e87510 admin admin] PATH_INFO: `/v3/projects/badc184402544a39868f96c9dd9c7f87/users/1bc4135eb1c74811bf31064218941bb2/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:00.014586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5e8bc-8d97-4b94-8c80-2226b3e87510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=badc184402544a39868f96c9dd9c7f87, user_id=1bc4135eb1c74811bf31064218941bb2, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:00.017469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec5e8bc-8d97-4b94-8c80-2226b3e87510 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:00.064810 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 851/1708] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:07:59 2022] PUT /identity/v3/projects/badc184402544a39868f96c9dd9c7f87/users/1bc4135eb1c74811bf31064218941bb2/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:00.079844 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ced32-00bc-4870-b098-0bf5f38b2b0d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:00.080216 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ced32-00bc-4870-b098-0bf5f38b2b0d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:00.080569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ced32-00bc-4870-b098-0bf5f38b2b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:00.149913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-b31ced32-00bc-4870-b098-0bf5f38b2b0d None None] MFA Rules not processed for user `1bc4135eb1c74811bf31064218941bb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:00.151395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b31ced32-00bc-4870-b098-0bf5f38b2b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:00.227535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:00.306608 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 858/1709] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:00.311424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5e8bc-8d97-4b94-8c80-2226b3e87510 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:00.498893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5e8bc-8d97-4b94-8c80-2226b3e87510 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:00.516025 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add2439-43dd-4e8a-bd60-7e535547573e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:00.517516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add2439-43dd-4e8a-bd60-7e535547573e admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:00.517798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add2439-43dd-4e8a-bd60-7e535547573e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:00.518161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add2439-43dd-4e8a-bd60-7e535547573e admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:00.518891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add2439-43dd-4e8a-bd60-7e535547573e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:00.521180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add2439-43dd-4e8a-bd60-7e535547573e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:00.588919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 852/1710] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:00 2022] POST /identity/v3/projects => generated 387 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:00.592984 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ced32-00bc-4870-b098-0bf5f38b2b0d None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:00.610916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ced32-00bc-4870-b098-0bf5f38b2b0d None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:00.779629 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbec763-1dfe-4364-9f85-bf83ea6fd730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:00.780462 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbec763-1dfe-4364-9f85-bf83ea6fd730 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:00.780769 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbec763-1dfe-4364-9f85-bf83ea6fd730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:00.781756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbec763-1dfe-4364-9f85-bf83ea6fd730 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:00.782672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbec763-1dfe-4364-9f85-bf83ea6fd730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:00.785692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbec763-1dfe-4364-9f85-bf83ea6fd730 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:00.833536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add2439-43dd-4e8a-bd60-7e535547573e admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:00.835338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4add2439-43dd-4e8a-bd60-7e535547573e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:00.847317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:00.861807 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 859/1711] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:00 2022] POST /identity/v3/users => generated 294 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:00.865786 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbec763-1dfe-4364-9f85-bf83ea6fd730 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:00.875923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add2439-43dd-4e8a-bd60-7e535547573e admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:00.885526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbec763-1dfe-4364-9f85-bf83ea6fd730 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:00.896560 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4febd-3db8-4d3b-9601-7a2fe36b357f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:00.897348 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4febd-3db8-4d3b-9601-7a2fe36b357f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:00.898171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4febd-3db8-4d3b-9601-7a2fe36b357f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:00.898594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4febd-3db8-4d3b-9601-7a2fe36b357f admin admin] PATH_INFO: `/v3/projects/f22c36df7bc547f6abd9533d0730a077` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:00.901887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afec31-151e-4f80-8807-6c97938d2b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:00.902572 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4febd-3db8-4d3b-9601-7a2fe36b357f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f22c36df7bc547f6abd9533d0730a077)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:00.902938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afec31-151e-4f80-8807-6c97938d2b44 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:00.903354 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afec31-151e-4f80-8807-6c97938d2b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:00.903800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37afec31-151e-4f80-8807-6c97938d2b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:00.904622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afec31-151e-4f80-8807-6c97938d2b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:00.906414 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37afec31-151e-4f80-8807-6c97938d2b44 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:00.910939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4febd-3db8-4d3b-9601-7a2fe36b357f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:00.935862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 860/1712] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:08:00.940963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afec31-151e-4f80-8807-6c97938d2b44 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:00.960002 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37afec31-151e-4f80-8807-6c97938d2b44 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:00.979861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344a2d8-74d4-4918-809a-1c328ba1c8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:00.980573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344a2d8-74d4-4918-809a-1c328ba1c8ef admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:00.980847 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344a2d8-74d4-4918-809a-1c328ba1c8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:00.981716 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344a2d8-74d4-4918-809a-1c328ba1c8ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:00.981838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344a2d8-74d4-4918-809a-1c328ba1c8ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:00.983403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344a2d8-74d4-4918-809a-1c328ba1c8ef admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:01.008004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 861/1713] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:01.010016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344a2d8-74d4-4918-809a-1c328ba1c8ef admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:01.028130 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344a2d8-74d4-4918-809a-1c328ba1c8ef admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:01.040301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b5c4febd-3db8-4d3b-9601-7a2fe36b357f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f22c36df7bc547f6abd9533d0730a077', 'request_id': 'req-b5c4febd-3db8-4d3b-9601-7a2fe36b357f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:01.060800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b5c4febd-3db8-4d3b-9601-7a2fe36b357f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f22c36df7bc547f6abd9533d0730a077', 'request_id': 'req-b5c4febd-3db8-4d3b-9601-7a2fe36b357f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:01.074748 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b5c4febd-3db8-4d3b-9601-7a2fe36b357f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f22c36df7bc547f6abd9533d0730a077', 'request_id': 'req-b5c4febd-3db8-4d3b-9601-7a2fe36b357f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:01.103372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b5c4febd-3db8-4d3b-9601-7a2fe36b357f admin admin] The token cache is being invalidate because project f22c36df7bc547f6abd9533d0730a077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:01.103834 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b5c4febd-3db8-4d3b-9601-7a2fe36b357f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:01.105928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b5c4febd-3db8-4d3b-9601-7a2fe36b357f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:01.108357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 853/1714] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:00 2022] DELETE /identity/v3/projects/f22c36df7bc547f6abd9533d0730a077 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:01.114233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933921b-6620-4327-84c8-40471a92d955 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:01.114724 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933921b-6620-4327-84c8-40471a92d955 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:01.115087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2933921b-6620-4327-84c8-40471a92d955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:01.150876 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-2933921b-6620-4327-84c8-40471a92d955 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:01.152620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-2933921b-6620-4327-84c8-40471a92d955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:01.159540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd281f90-afe3-4a0b-8622-fcaa7843ca43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:01.160232 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd281f90-afe3-4a0b-8622-fcaa7843ca43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:01.160507 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd281f90-afe3-4a0b-8622-fcaa7843ca43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:01.160783 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd281f90-afe3-4a0b-8622-fcaa7843ca43 admin admin] PATH_INFO: `/v3/projects/4753b7d382494a92a44700f226519147/users/0deb38c37fca47afab4cb5686ad9d4cf/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:01.166376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 854/1715] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:01.176202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c006ef-166f-4a54-a1ea-848041894f22 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:01.176560 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c006ef-166f-4a54-a1ea-848041894f22 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:01.176923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c006ef-166f-4a54-a1ea-848041894f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:01.205478 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd281f90-afe3-4a0b-8622-fcaa7843ca43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4753b7d382494a92a44700f226519147, user_id=0deb38c37fca47afab4cb5686ad9d4cf, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:01.207860 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd281f90-afe3-4a0b-8622-fcaa7843ca43 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:01.233780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 862/1716] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:01 2022] PUT /identity/v3/projects/4753b7d382494a92a44700f226519147/users/0deb38c37fca47afab4cb5686ad9d4cf/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:01.240746 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-f5c006ef-166f-4a54-a1ea-848041894f22 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:01.244818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527acc2-cf56-40fa-b183-7a578e2adee9 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:01.245317 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527acc2-cf56-40fa-b183-7a578e2adee9 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:01.245693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f5c006ef-166f-4a54-a1ea-848041894f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:01.246011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527acc2-cf56-40fa-b183-7a578e2adee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:01.307969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-6527acc2-cf56-40fa-b183-7a578e2adee9 None None] MFA Rules not processed for user `0deb38c37fca47afab4cb5686ad9d4cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:01.309598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6527acc2-cf56-40fa-b183-7a578e2adee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:01.337637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:01.429915 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 855/1717] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:01.434189 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c006ef-166f-4a54-a1ea-848041894f22 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:01.462697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c006ef-166f-4a54-a1ea-848041894f22 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:01.474904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766bda3c-155f-4390-a058-1ad4b2223d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:01.476479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766bda3c-155f-4390-a058-1ad4b2223d33 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:01.477702 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766bda3c-155f-4390-a058-1ad4b2223d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:01.478899 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766bda3c-155f-4390-a058-1ad4b2223d33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:01.479604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766bda3c-155f-4390-a058-1ad4b2223d33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:01.480797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766bda3c-155f-4390-a058-1ad4b2223d33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:01.482831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766bda3c-155f-4390-a058-1ad4b2223d33 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:01.511270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 856/1718] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:08:01.515982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766bda3c-155f-4390-a058-1ad4b2223d33 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:01.517728 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 863/1719] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:01.528742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527acc2-cf56-40fa-b183-7a578e2adee9 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:01.537931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766bda3c-155f-4390-a058-1ad4b2223d33 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:01.540062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6527acc2-cf56-40fa-b183-7a578e2adee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:01.550270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18291e62-7943-4f73-bc78-ca1261da5971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:01.550995 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18291e62-7943-4f73-bc78-ca1261da5971 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:01.551342 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18291e62-7943-4f73-bc78-ca1261da5971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:01.552536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18291e62-7943-4f73-bc78-ca1261da5971 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:01.553287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18291e62-7943-4f73-bc78-ca1261da5971 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:01.555307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18291e62-7943-4f73-bc78-ca1261da5971 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:01.564072 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527acc2-cf56-40fa-b183-7a578e2adee9 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:01.587473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399de1dc-0597-46ed-a2f0-650adc72a0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:01.587473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399de1dc-0597-46ed-a2f0-650adc72a0eb admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:01.587473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399de1dc-0597-46ed-a2f0-650adc72a0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:01.587473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399de1dc-0597-46ed-a2f0-650adc72a0eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:01.587933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399de1dc-0597-46ed-a2f0-650adc72a0eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:01.587933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399de1dc-0597-46ed-a2f0-650adc72a0eb admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:01.626768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 857/1720] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:01 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:01.630892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18291e62-7943-4f73-bc78-ca1261da5971 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:01.673317 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 864/1721] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:01 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:01.676132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399de1dc-0597-46ed-a2f0-650adc72a0eb admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:01.791854 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:01.832700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399de1dc-0597-46ed-a2f0-650adc72a0eb admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:01.845752 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37816e-d905-404d-b652-9f382afc9d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:01.846071 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18291e62-7943-4f73-bc78-ca1261da5971 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:01.847363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37816e-d905-404d-b652-9f382afc9d45 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:01.848039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37816e-d905-404d-b652-9f382afc9d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:01.848154 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37816e-d905-404d-b652-9f382afc9d45 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:01.849277 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37816e-d905-404d-b652-9f382afc9d45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:01.854308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37816e-d905-404d-b652-9f382afc9d45 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:01.873561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe80ece-6099-4450-910a-26b429be973d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:01.874304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe80ece-6099-4450-910a-26b429be973d admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:01.876352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe80ece-6099-4450-910a-26b429be973d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:01.876776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe80ece-6099-4450-910a-26b429be973d admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:01.877438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe80ece-6099-4450-910a-26b429be973d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:01.882153 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe80ece-6099-4450-910a-26b429be973d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:01.903481 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 865/1722] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:01 2022] POST /identity/v3/users => generated 300 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:01.908929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37816e-d905-404d-b652-9f382afc9d45 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:01.927480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37816e-d905-404d-b652-9f382afc9d45 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:01.942607 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1018e77-4623-4e2e-9deb-2a87f4afa579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:01.945453 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1018e77-4623-4e2e-9deb-2a87f4afa579 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:01.948149 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1018e77-4623-4e2e-9deb-2a87f4afa579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:01.948658 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1018e77-4623-4e2e-9deb-2a87f4afa579 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:01.950348 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1018e77-4623-4e2e-9deb-2a87f4afa579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:01.956282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 858/1723] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:01 2022] POST /identity/v3/users => generated 286 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:01.960196 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1018e77-4623-4e2e-9deb-2a87f4afa579 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:01.960603 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe80ece-6099-4450-910a-26b429be973d admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:01.980859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe80ece-6099-4450-910a-26b429be973d admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:01.986772 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849f2b7-52fb-4907-9e4f-ee99de0c46a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:01.987603 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849f2b7-52fb-4907-9e4f-ee99de0c46a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:01.987756 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849f2b7-52fb-4907-9e4f-ee99de0c46a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:01.988105 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c849f2b7-52fb-4907-9e4f-ee99de0c46a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:01.989247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849f2b7-52fb-4907-9e4f-ee99de0c46a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:01.991056 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c849f2b7-52fb-4907-9e4f-ee99de0c46a1 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:02.010767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 866/1724] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:02.011991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1018e77-4623-4e2e-9deb-2a87f4afa579 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:02.015341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 859/1725] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:02.019835 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849f2b7-52fb-4907-9e4f-ee99de0c46a1 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:02.037428 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c849f2b7-52fb-4907-9e4f-ee99de0c46a1 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:02.053166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2f02e-373d-464e-8850-cb0bde9562a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:02.053663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2f02e-373d-464e-8850-cb0bde9562a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:02.053946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2f02e-373d-464e-8850-cb0bde9562a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:02.054046 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1018e77-4623-4e2e-9deb-2a87f4afa579 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:02.054211 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2f02e-373d-464e-8850-cb0bde9562a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:02.054877 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2f02e-373d-464e-8850-cb0bde9562a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:02.057700 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a2f02e-373d-464e-8850-cb0bde9562a1 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:02.072529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2db706-149d-478d-bac8-766d8ca3beb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:02.073269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2db706-149d-478d-bac8-766d8ca3beb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:02.073678 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2db706-149d-478d-bac8-766d8ca3beb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:02.074129 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2db706-149d-478d-bac8-766d8ca3beb9 admin admin] PATH_INFO: `/v3/projects/585945b6362b4f47b19019499cedbcdf/users/713f610eec854e60a8fb3edb0adc227f/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:02.103260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 860/1726] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:02.108880 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2f02e-373d-464e-8850-cb0bde9562a1 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:02.130697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2f02e-373d-464e-8850-cb0bde9562a1 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:02.145691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2db706-149d-478d-bac8-766d8ca3beb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=585945b6362b4f47b19019499cedbcdf, user_id=713f610eec854e60a8fb3edb0adc227f, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:02.156037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598011d2-5261-4a0a-b982-c53c07f79337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:02.158912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598011d2-5261-4a0a-b982-c53c07f79337 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:02.159373 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598011d2-5261-4a0a-b982-c53c07f79337 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:02.159506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598011d2-5261-4a0a-b982-c53c07f79337 admin admin] PATH_INFO: `/v3/projects/1f291db8b4964794928b05f3a19f053f/users/917b548bcbc44174931775eaf6999d60/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:02.162502 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2db706-149d-478d-bac8-766d8ca3beb9 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:02.219418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598011d2-5261-4a0a-b982-c53c07f79337 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f291db8b4964794928b05f3a19f053f, user_id=917b548bcbc44174931775eaf6999d60, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:02.225580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 867/1727] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:02 2022] PUT /identity/v3/projects/585945b6362b4f47b19019499cedbcdf/users/713f610eec854e60a8fb3edb0adc227f/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:02.229601 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2db706-149d-478d-bac8-766d8ca3beb9 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:02.231992 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598011d2-5261-4a0a-b982-c53c07f79337 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:02.291071 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 861/1728] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:02 2022] PUT /identity/v3/projects/1f291db8b4964794928b05f3a19f053f/users/917b548bcbc44174931775eaf6999d60/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:02.303034 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485994c-bfe7-4c2a-994c-d29cc8bb7b5e None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:02.303378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485994c-bfe7-4c2a-994c-d29cc8bb7b5e None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:02.303632 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c485994c-bfe7-4c2a-994c-d29cc8bb7b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:02.394301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:02.401281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-c485994c-bfe7-4c2a-994c-d29cc8bb7b5e None None] MFA Rules not processed for user `917b548bcbc44174931775eaf6999d60`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:02.402740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c485994c-bfe7-4c2a-994c-d29cc8bb7b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:02.443654 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2db706-149d-478d-bac8-766d8ca3beb9 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:02.567205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb7427-5936-46a4-b54b-bab27c928aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:02.568246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb7427-5936-46a4-b54b-bab27c928aed admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:02.568741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb7427-5936-46a4-b54b-bab27c928aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:02.569203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bb7427-5936-46a4-b54b-bab27c928aed admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:02.570050 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb7427-5936-46a4-b54b-bab27c928aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:02.573493 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bb7427-5936-46a4-b54b-bab27c928aed admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:02.589255 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 862/1729] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:02.593447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485994c-bfe7-4c2a-994c-d29cc8bb7b5e None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:02.605693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 868/1730] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:08:02.606889 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb7427-5936-46a4-b54b-bab27c928aed admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:02.621587 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c485994c-bfe7-4c2a-994c-d29cc8bb7b5e None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:02.629988 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bb7427-5936-46a4-b54b-bab27c928aed admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:02.630274 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d64694-6c1c-4959-87ff-887d639829e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:02.631006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d64694-6c1c-4959-87ff-887d639829e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:02.631392 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d64694-6c1c-4959-87ff-887d639829e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:02.631584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d64694-6c1c-4959-87ff-887d639829e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:02.632235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d64694-6c1c-4959-87ff-887d639829e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:02.634636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d64694-6c1c-4959-87ff-887d639829e9 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:02.642799 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2ab9ab-5646-48a4-9c82-ab2004d11917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:02.643759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ab9ab-5646-48a4-9c82-ab2004d11917 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:02.644203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ab9ab-5646-48a4-9c82-ab2004d11917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:02.644519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2ab9ab-5646-48a4-9c82-ab2004d11917 admin admin] PATH_INFO: `/v3/projects/585945b6362b4f47b19019499cedbcdf/users/713f610eec854e60a8fb3edb0adc227f/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:02.648696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ab9ab-5646-48a4-9c82-ab2004d11917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=585945b6362b4f47b19019499cedbcdf, user_id=713f610eec854e60a8fb3edb0adc227f, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:02.651792 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2ab9ab-5646-48a4-9c82-ab2004d11917 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:02.687203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 869/1731] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:02 2022] PUT /identity/v3/projects/585945b6362b4f47b19019499cedbcdf/users/713f610eec854e60a8fb3edb0adc227f/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:02.692487 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150276a0-b895-47e8-b318-b69b27b02924 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:02.692881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150276a0-b895-47e8-b318-b69b27b02924 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:02.693256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150276a0-b895-47e8-b318-b69b27b02924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:02.726395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 863/1732] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:02 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:02.728676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7883da-c7cb-4989-a410-baa6c313fc61 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:02.728676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7883da-c7cb-4989-a410-baa6c313fc61 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:02.728676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7883da-c7cb-4989-a410-baa6c313fc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:02.746055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-150276a0-b895-47e8-b318-b69b27b02924 None None] MFA Rules not processed for user `713f610eec854e60a8fb3edb0adc227f`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:02.748297 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-150276a0-b895-47e8-b318-b69b27b02924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:02.816080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-2d7883da-c7cb-4989-a410-baa6c313fc61 None None] MFA Rules not processed for user `675b1a5759844cdaa7e0bf071a3ee030`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:02.819271 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-2d7883da-c7cb-4989-a410-baa6c313fc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:02.934147 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 870/1733] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:02.935542 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150276a0-b895-47e8-b318-b69b27b02924 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:02.939523 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:03.003577 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 864/1734] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:03.003577 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7883da-c7cb-4989-a410-baa6c313fc61 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:03.005410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-2d7883da-c7cb-4989-a410-baa6c313fc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:03.095116 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:03.103092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7883da-c7cb-4989-a410-baa6c313fc61 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:03.105786 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150276a0-b895-47e8-b318-b69b27b02924 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:03.120567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84b155-469d-41c3-b7a3-12af580c85fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:03.122246 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84b155-469d-41c3-b7a3-12af580c85fb service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:03.122736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84b155-469d-41c3-b7a3-12af580c85fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:03.123201 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d84b155-469d-41c3-b7a3-12af580c85fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:03.125873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf281a-2238-434a-b150-978d8d9bbfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:03.126510 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf281a-2238-434a-b150-978d8d9bbfe5 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:03.126979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf281a-2238-434a-b150-978d8d9bbfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:03.127187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf281a-2238-434a-b150-978d8d9bbfe5 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:03.127864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf281a-2238-434a-b150-978d8d9bbfe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:03.135205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abf281a-2238-434a-b150-978d8d9bbfe5 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:03.149847 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84b155-469d-41c3-b7a3-12af580c85fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:03.153174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d84b155-469d-41c3-b7a3-12af580c85fb service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:03.201656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 871/1735] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:02 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:03.202954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76ba36-5640-4826-8742-2b77cf727ec9 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:03.203238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76ba36-5640-4826-8742-2b77cf727ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:03.204861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76ba36-5640-4826-8742-2b77cf727ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:03.219803 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 865/1736] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:03.221192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84b155-469d-41c3-b7a3-12af580c85fb service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:03.256753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-fd76ba36-5640-4826-8742-2b77cf727ec9 None None] MFA Rules not processed for user `675b1a5759844cdaa7e0bf071a3ee030`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:03.258650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-fd76ba36-5640-4826-8742-2b77cf727ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:03.284059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 872/1737] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:03.288365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d84b155-469d-41c3-b7a3-12af580c85fb service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:03.296829 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76ba36-5640-4826-8742-2b77cf727ec9 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:03.298817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-fd76ba36-5640-4826-8742-2b77cf727ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:03.309589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd2025-afd3-4425-8d8b-6c80c31006a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:03.316612 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd2025-afd3-4425-8d8b-6c80c31006a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:03.317447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd2025-afd3-4425-8d8b-6c80c31006a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:03.320848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fd2025-afd3-4425-8d8b-6c80c31006a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:03.324834 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd2025-afd3-4425-8d8b-6c80c31006a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:03.334266 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fd2025-afd3-4425-8d8b-6c80c31006a2 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:03.379098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 866/1738] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:03.380833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd2025-afd3-4425-8d8b-6c80c31006a2 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:03.455905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fd2025-afd3-4425-8d8b-6c80c31006a2 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:03.476732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a8771-ee10-446e-a73b-6b6081b5a8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:03.481106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a8771-ee10-446e-a73b-6b6081b5a8e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:03.483263 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a8771-ee10-446e-a73b-6b6081b5a8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:03.483725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a8771-ee10-446e-a73b-6b6081b5a8e5 admin admin] PATH_INFO: `/v3/projects/221f816e91c04414b1fbb27dacc0bb41/users/204b90d5623c4d8e80e7748c0883f142/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:03.561731 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a8771-ee10-446e-a73b-6b6081b5a8e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=221f816e91c04414b1fbb27dacc0bb41, user_id=204b90d5623c4d8e80e7748c0883f142, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:03.567915 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a8771-ee10-446e-a73b-6b6081b5a8e5 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:03.579197 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76ba36-5640-4826-8742-2b77cf727ec9 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:03.587613 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf512bd0-5c5f-4b6f-b185-d6e729d386e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:03.588605 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf512bd0-5c5f-4b6f-b185-d6e729d386e6 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:03.589114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf512bd0-5c5f-4b6f-b185-d6e729d386e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:03.589607 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf512bd0-5c5f-4b6f-b185-d6e729d386e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:03.611704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 867/1739] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:03 2022] PUT /identity/v3/projects/221f816e91c04414b1fbb27dacc0bb41/users/204b90d5623c4d8e80e7748c0883f142/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:03.614830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a8771-ee10-446e-a73b-6b6081b5a8e5 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:03.617719 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf512bd0-5c5f-4b6f-b185-d6e729d386e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:03.619845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf512bd0-5c5f-4b6f-b185-d6e729d386e6 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:03.636023 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 873/1740] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:03.882844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a8771-ee10-446e-a73b-6b6081b5a8e5 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:03.911066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109bc12-9c46-4552-b3fe-5ebeca36f367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:03.911749 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109bc12-9c46-4552-b3fe-5ebeca36f367 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:03.912040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109bc12-9c46-4552-b3fe-5ebeca36f367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:03.912309 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109bc12-9c46-4552-b3fe-5ebeca36f367 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:03.917035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109bc12-9c46-4552-b3fe-5ebeca36f367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:03.917491 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109bc12-9c46-4552-b3fe-5ebeca36f367 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:03.953473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 868/1741] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:03.960652 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf512bd0-5c5f-4b6f-b185-d6e729d386e6 service nova] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:04.035602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf512bd0-5c5f-4b6f-b185-d6e729d386e6 service nova] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:04.061273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cecac-a6df-42fc-8b14-8c2f04542e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:04.063356 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cecac-a6df-42fc-8b14-8c2f04542e8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:04.063868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cecac-a6df-42fc-8b14-8c2f04542e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:04.066895 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0cecac-a6df-42fc-8b14-8c2f04542e8f admin admin] PATH_INFO: `/v3/projects/221f816e91c04414b1fbb27dacc0bb41/users/204b90d5623c4d8e80e7748c0883f142/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:04.069367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cecac-a6df-42fc-8b14-8c2f04542e8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=221f816e91c04414b1fbb27dacc0bb41, user_id=204b90d5623c4d8e80e7748c0883f142, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:04.074984 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0cecac-a6df-42fc-8b14-8c2f04542e8f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:04.087148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 874/1742] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:03 2022] PUT /identity/v3/projects/221f816e91c04414b1fbb27dacc0bb41/users/204b90d5623c4d8e80e7748c0883f142/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:04.098849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee6aee-0bfb-4c77-a88c-0a42ca57df64 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:04.098849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee6aee-0bfb-4c77-a88c-0a42ca57df64 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:04.098849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee6aee-0bfb-4c77-a88c-0a42ca57df64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:04.153569 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-c0ee6aee-0bfb-4c77-a88c-0a42ca57df64 None None] MFA Rules not processed for user `204b90d5623c4d8e80e7748c0883f142`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:04.155375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c0ee6aee-0bfb-4c77-a88c-0a42ca57df64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:04.276949 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:04.374234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 869/1743] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:04.379666 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cecac-a6df-42fc-8b14-8c2f04542e8f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:04.381468 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ad0cecac-a6df-42fc-8b14-8c2f04542e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:04.504385 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0cecac-a6df-42fc-8b14-8c2f04542e8f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:04.508838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edbae0-3a56-442f-a871-ee152470fc0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:04.509721 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edbae0-3a56-442f-a871-ee152470fc0f service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:04.510598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edbae0-3a56-442f-a871-ee152470fc0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:04.511045 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8edbae0-3a56-442f-a871-ee152470fc0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:04.527577 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edbae0-3a56-442f-a871-ee152470fc0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:04.529586 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8edbae0-3a56-442f-a871-ee152470fc0f service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:04.553465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 875/1744] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:04.726759 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee6aee-0bfb-4c77-a88c-0a42ca57df64 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:04.750640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee6aee-0bfb-4c77-a88c-0a42ca57df64 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:04.762561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9aaace-336a-42af-a5cf-ebdf85514b3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:04.763801 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9aaace-336a-42af-a5cf-ebdf85514b3f service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:04.764189 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9aaace-336a-42af-a5cf-ebdf85514b3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:04.764601 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9aaace-336a-42af-a5cf-ebdf85514b3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:04.961020 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9aaace-336a-42af-a5cf-ebdf85514b3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:04.965851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9aaace-336a-42af-a5cf-ebdf85514b3f service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:04.994417 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edbae0-3a56-442f-a871-ee152470fc0f service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:05.004909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8edbae0-3a56-442f-a871-ee152470fc0f service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:05.021995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693ac7f-1121-4bfe-ba9c-f64372fe7976 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:05.022835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693ac7f-1121-4bfe-ba9c-f64372fe7976 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:05.023248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 870/1745] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:05.023794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693ac7f-1121-4bfe-ba9c-f64372fe7976 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:05.024513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693ac7f-1121-4bfe-ba9c-f64372fe7976 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:05.029541 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6693ac7f-1121-4bfe-ba9c-f64372fe7976 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:05.155897 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:05.166803 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693ac7f-1121-4bfe-ba9c-f64372fe7976 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:05.168673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693ac7f-1121-4bfe-ba9c-f64372fe7976 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:05.193339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eaf19-d6c3-492a-9295-6309c4be955e None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:05.197041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eaf19-d6c3-492a-9295-6309c4be955e None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:05.197424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897eaf19-d6c3-492a-9295-6309c4be955e None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:05.201205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 871/1746] 10.222.0.27 () {54 vars in 912 bytes} [Thu Sep 8 16:08:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:08:05.206120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee272f-ad72-4ade-8c08-93f5a660988b None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:05.206515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee272f-ad72-4ade-8c08-93f5a660988b None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:05.206884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee272f-ad72-4ade-8c08-93f5a660988b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:05.227722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 876/1747] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:05.368519 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-eeee272f-ad72-4ade-8c08-93f5a660988b None None] MFA Rules not processed for user `3aae65f9ec394f97aeafad43932dc49c`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:05.369796 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-eeee272f-ad72-4ade-8c08-93f5a660988b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:05.612371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 872/1748] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Sep 8 16:08:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:05.666492 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693ac7f-1121-4bfe-ba9c-f64372fe7976 service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:05.668159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6693ac7f-1121-4bfe-ba9c-f64372fe7976 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:05.690881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693ac7f-1121-4bfe-ba9c-f64372fe7976 service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:05.708089 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71564249-87c2-4cbf-9515-e97ce8f6e309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:05.709214 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71564249-87c2-4cbf-9515-e97ce8f6e309 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:05.709496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71564249-87c2-4cbf-9515-e97ce8f6e309 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:05.713364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71564249-87c2-4cbf-9515-e97ce8f6e309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:05.750600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71564249-87c2-4cbf-9515-e97ce8f6e309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:05.754754 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71564249-87c2-4cbf-9515-e97ce8f6e309 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:05.766083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 877/1749] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:06.352485 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b572876-d1b5-46fd-b522-1ef772c913c6 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:06.353031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b572876-d1b5-46fd-b522-1ef772c913c6 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:06.354002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b572876-d1b5-46fd-b522-1ef772c913c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:06.470300 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-0b572876-d1b5-46fd-b522-1ef772c913c6 None None] MFA Rules not processed for user `00b673d5c65d40d9a90cc3b579e83e7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:06.471900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0b572876-d1b5-46fd-b522-1ef772c913c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:06.625578 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:06.645116 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 873/1750] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:06.672180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71564249-87c2-4cbf-9515-e97ce8f6e309 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:06.673868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-71564249-87c2-4cbf-9515-e97ce8f6e309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:06.924612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71564249-87c2-4cbf-9515-e97ce8f6e309 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:06.942410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bdc47-bb54-4f19-9092-0f7448128026 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:06.943518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bdc47-bb54-4f19-9092-0f7448128026 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:06.944166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bdc47-bb54-4f19-9092-0f7448128026 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:06.944647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bdc47-bb54-4f19-9092-0f7448128026 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:06.961971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8bdc47-bb54-4f19-9092-0f7448128026 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:06.965559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8bdc47-bb54-4f19-9092-0f7448128026 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:06.991497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 878/1751] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:08:07.859333 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b572876-d1b5-46fd-b522-1ef772c913c6 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:07.860876 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0b572876-d1b5-46fd-b522-1ef772c913c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:08.000492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b572876-d1b5-46fd-b522-1ef772c913c6 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:08.038547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a1ee3c-e8e0-4009-a85a-6521f5b40337 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd28bb31b55e45668212c9ab69ed3ac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:08.039986 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1ee3c-e8e0-4009-a85a-6521f5b40337 service placement] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:08.040293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1ee3c-e8e0-4009-a85a-6521f5b40337 service placement] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:08.040558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a1ee3c-e8e0-4009-a85a-6521f5b40337 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:08.047473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-60a1ee3c-e8e0-4009-a85a-6521f5b40337 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:08.101420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1ee3c-e8e0-4009-a85a-6521f5b40337 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:08.103472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a1ee3c-e8e0-4009-a85a-6521f5b40337 service placement] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:08.133378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 874/1752] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:08.207087 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50092e22-e568-4095-834c-24bae8b2be8c None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:08.207736 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50092e22-e568-4095-834c-24bae8b2be8c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:08.207736 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50092e22-e568-4095-834c-24bae8b2be8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:08.330490 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-50092e22-e568-4095-834c-24bae8b2be8c None None] MFA Rules not processed for user `18375266a64348df879a9eec53bfcbf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:08.332855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-50092e22-e568-4095-834c-24bae8b2be8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:08.540344 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 879/1753] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:08.551132 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebba280-f2ab-42ad-9c38-37169e3dafc7 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:08.551727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebba280-f2ab-42ad-9c38-37169e3dafc7 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:08.553268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebba280-f2ab-42ad-9c38-37169e3dafc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:08.586374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-6ebba280-f2ab-42ad-9c38-37169e3dafc7 None None] MFA Rules not processed for user `18375266a64348df879a9eec53bfcbf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:08.587731 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6ebba280-f2ab-42ad-9c38-37169e3dafc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:08.606615 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 875/1754] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:08.618539 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50092e22-e568-4095-834c-24bae8b2be8c None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:08.638324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50092e22-e568-4095-834c-24bae8b2be8c None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:08.657761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df4bc7-bec6-4a97-864e-7229842ddd89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:08.661022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df4bc7-bec6-4a97-864e-7229842ddd89 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:08.661316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df4bc7-bec6-4a97-864e-7229842ddd89 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:08.663058 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df4bc7-bec6-4a97-864e-7229842ddd89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:08.685731 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df4bc7-bec6-4a97-864e-7229842ddd89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:08.688137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df4bc7-bec6-4a97-864e-7229842ddd89 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:08.706452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 880/1755] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:08 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:09.431112 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6896b54-8248-4aaa-aaf9-62656f187d53 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:09.431654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6896b54-8248-4aaa-aaf9-62656f187d53 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:09.431654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6896b54-8248-4aaa-aaf9-62656f187d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:09.509809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-c6896b54-8248-4aaa-aaf9-62656f187d53 None None] MFA Rules not processed for user `c89859a30a2442b8ae13fe0444782d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:09.515405 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c6896b54-8248-4aaa-aaf9-62656f187d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:09.734547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 876/1756] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:09.744759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df4bc7-bec6-4a97-864e-7229842ddd89 service nova] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:09.778893 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df4bc7-bec6-4a97-864e-7229842ddd89 service nova] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:09.801154 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f79b7d-57c1-4237-9322-54ba1ae61d9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:09.804410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f79b7d-57c1-4237-9322-54ba1ae61d9f service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:09.804858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f79b7d-57c1-4237-9322-54ba1ae61d9f service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:09.805585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f79b7d-57c1-4237-9322-54ba1ae61d9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:09.832529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f79b7d-57c1-4237-9322-54ba1ae61d9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:09.835741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f79b7d-57c1-4237-9322-54ba1ae61d9f service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:09.867906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 881/1757] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:09 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:10.468625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6896b54-8248-4aaa-aaf9-62656f187d53 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:10.472905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c6896b54-8248-4aaa-aaf9-62656f187d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:10.697379 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6896b54-8248-4aaa-aaf9-62656f187d53 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:10.716246 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4961e87-34b7-447a-8138-ffce5898a151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:10.718459 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4961e87-34b7-447a-8138-ffce5898a151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:10.718751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4961e87-34b7-447a-8138-ffce5898a151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:10.719020 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4961e87-34b7-447a-8138-ffce5898a151 admin admin] PATH_INFO: `/v3/users/a7102fe9b2654a079125597e49846614` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:10.723193 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4961e87-34b7-447a-8138-ffce5898a151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7102fe9b2654a079125597e49846614)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:10.731109 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4961e87-34b7-447a-8138-ffce5898a151 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:10.762076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:10.876792 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c4961e87-34b7-447a-8138-ffce5898a151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7102fe9b2654a079125597e49846614'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:10.890840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c4961e87-34b7-447a-8138-ffce5898a151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7102fe9b2654a079125597e49846614'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:10.891958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c4961e87-34b7-447a-8138-ffce5898a151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7102fe9b2654a079125597e49846614'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:10.892940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c4961e87-34b7-447a-8138-ffce5898a151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7102fe9b2654a079125597e49846614'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:10.906164 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c4961e87-34b7-447a-8138-ffce5898a151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7102fe9b2654a079125597e49846614'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:10.913612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9699ae-6cfa-481f-aa3b-c1e7b1289ebb None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:10.914152 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9699ae-6cfa-481f-aa3b-c1e7b1289ebb None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:10.914224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9699ae-6cfa-481f-aa3b-c1e7b1289ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:10.964650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-8e9699ae-6cfa-481f-aa3b-c1e7b1289ebb None None] MFA Rules not processed for user `18375266a64348df879a9eec53bfcbf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:10.965896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8e9699ae-6cfa-481f-aa3b-c1e7b1289ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:10.971147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 877/1758] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:10 2022] DELETE /identity/v3/users/a7102fe9b2654a079125597e49846614 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:10.975986 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add40d98-bf3d-4eb5-8fd8-e76708aaa446 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:10.976412 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add40d98-bf3d-4eb5-8fd8-e76708aaa446 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:10.976830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add40d98-bf3d-4eb5-8fd8-e76708aaa446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:11.028664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-add40d98-bf3d-4eb5-8fd8-e76708aaa446 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:11.031056 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-add40d98-bf3d-4eb5-8fd8-e76708aaa446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:11.052896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:11.162945 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 882/1759] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:11.203360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 878/1760] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:11.565087 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9699ae-6cfa-481f-aa3b-c1e7b1289ebb None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:11.566924 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8e9699ae-6cfa-481f-aa3b-c1e7b1289ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:11.704266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9699ae-6cfa-481f-aa3b-c1e7b1289ebb None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:11.717655 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a0c50-3d23-4cab-95ee-5613d3bcf822 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:11.718727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a0c50-3d23-4cab-95ee-5613d3bcf822 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:11.719152 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a0c50-3d23-4cab-95ee-5613d3bcf822 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:11.719767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a0c50-3d23-4cab-95ee-5613d3bcf822 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:11.747409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a0c50-3d23-4cab-95ee-5613d3bcf822 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:11.748568 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a0c50-3d23-4cab-95ee-5613d3bcf822 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:11.769740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 883/1761] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:12.680614 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00871b-f229-4a81-85fc-32eb7dd0f754 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:12.681186 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00871b-f229-4a81-85fc-32eb7dd0f754 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:12.681259 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00871b-f229-4a81-85fc-32eb7dd0f754 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:12.683038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 879/1762] 10.222.0.27 () {54 vars in 912 bytes} [Thu Sep 8 16:08:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:08:12.689047 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae4a25-0dc8-452e-8f6e-5b5b1dccb604 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:12.689537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae4a25-0dc8-452e-8f6e-5b5b1dccb604 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:12.689630 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae4a25-0dc8-452e-8f6e-5b5b1dccb604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:12.744813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-afae4a25-0dc8-452e-8f6e-5b5b1dccb604 None None] MFA Rules not processed for user `3aae65f9ec394f97aeafad43932dc49c`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:12.747329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-afae4a25-0dc8-452e-8f6e-5b5b1dccb604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:12.787623 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 884/1763] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Sep 8 16:08:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:12.812555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00871b-f229-4a81-85fc-32eb7dd0f754 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:12.836228 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00871b-f229-4a81-85fc-32eb7dd0f754 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:12.847796 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a932054f-6937-4dfb-b692-035dfc91e60d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:12.848443 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932054f-6937-4dfb-b692-035dfc91e60d service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:12.848720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932054f-6937-4dfb-b692-035dfc91e60d service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:12.848991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a932054f-6937-4dfb-b692-035dfc91e60d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:12.863034 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932054f-6937-4dfb-b692-035dfc91e60d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:12.879191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a932054f-6937-4dfb-b692-035dfc91e60d service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:12.908033 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 880/1764] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:13.107972 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407fe1af-6bf6-45b9-a414-b9cf6f654f32 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:13.108208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407fe1af-6bf6-45b9-a414-b9cf6f654f32 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:13.108980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407fe1af-6bf6-45b9-a414-b9cf6f654f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:13.186900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-407fe1af-6bf6-45b9-a414-b9cf6f654f32 None None] MFA Rules not processed for user `00b673d5c65d40d9a90cc3b579e83e7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:13.191693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-407fe1af-6bf6-45b9-a414-b9cf6f654f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:13.329105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 885/1765] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:13.435907 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461efc1-6ac6-45eb-a304-baafb5aae453 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:13.436496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461efc1-6ac6-45eb-a304-baafb5aae453 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:13.437889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b461efc1-6ac6-45eb-a304-baafb5aae453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:13.502172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-b461efc1-6ac6-45eb-a304-baafb5aae453 None None] MFA Rules not processed for user `c89859a30a2442b8ae13fe0444782d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:13.503928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b461efc1-6ac6-45eb-a304-baafb5aae453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:13.586945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 881/1766] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:13.795742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407fe1af-6bf6-45b9-a414-b9cf6f654f32 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:13.800076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-407fe1af-6bf6-45b9-a414-b9cf6f654f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:14.063814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407fe1af-6bf6-45b9-a414-b9cf6f654f32 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:14.085461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae15c23-797f-4623-ada0-7f20a4af691c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:14.087954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae15c23-797f-4623-ada0-7f20a4af691c service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:14.090069 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae15c23-797f-4623-ada0-7f20a4af691c service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:14.090538 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae15c23-797f-4623-ada0-7f20a4af691c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:14.096857 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0ae15c23-797f-4623-ada0-7f20a4af691c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:14.143117 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:14.175508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461efc1-6ac6-45eb-a304-baafb5aae453 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:14.189802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b461efc1-6ac6-45eb-a304-baafb5aae453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:14.232129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b461efc1-6ac6-45eb-a304-baafb5aae453 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:14.267061 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae15c23-797f-4623-ada0-7f20a4af691c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:14.272600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae15c23-797f-4623-ada0-7f20a4af691c service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:14.272765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a70014-1817-4e02-8028-0149bbd46a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:14.273358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a70014-1817-4e02-8028-0149bbd46a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:14.277091 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a70014-1817-4e02-8028-0149bbd46a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:14.277559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a70014-1817-4e02-8028-0149bbd46a2e admin admin] PATH_INFO: `/v3/projects/836c5986c96e424a8d4b8b0e26c0ddc1` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:14.283046 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a70014-1817-4e02-8028-0149bbd46a2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=836c5986c96e424a8d4b8b0e26c0ddc1)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:14.286860 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a70014-1817-4e02-8028-0149bbd46a2e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:14.318810 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 886/1767] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:13 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:14.369548 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:14.484492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-07a70014-1817-4e02-8028-0149bbd46a2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '836c5986c96e424a8d4b8b0e26c0ddc1', 'request_id': 'req-07a70014-1817-4e02-8028-0149bbd46a2e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:14.502988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-07a70014-1817-4e02-8028-0149bbd46a2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '836c5986c96e424a8d4b8b0e26c0ddc1', 'request_id': 'req-07a70014-1817-4e02-8028-0149bbd46a2e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:14.515793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789579c-1537-4749-b82a-3dc973f77966 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:14.516193 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789579c-1537-4749-b82a-3dc973f77966 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:14.516597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789579c-1537-4749-b82a-3dc973f77966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:14.517245 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-07a70014-1817-4e02-8028-0149bbd46a2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '836c5986c96e424a8d4b8b0e26c0ddc1', 'request_id': 'req-07a70014-1817-4e02-8028-0149bbd46a2e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:14.533711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-07a70014-1817-4e02-8028-0149bbd46a2e admin admin] The token cache is being invalidate because project 836c5986c96e424a8d4b8b0e26c0ddc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:14.534303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-07a70014-1817-4e02-8028-0149bbd46a2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:14.535768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-07a70014-1817-4e02-8028-0149bbd46a2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:14.546075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 882/1768] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:14 2022] DELETE /identity/v3/projects/836c5986c96e424a8d4b8b0e26c0ddc1 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:14.547307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a70014-1817-4e02-8028-0149bbd46a2e admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:14.548983 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-07a70014-1817-4e02-8028-0149bbd46a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:14.651919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-a789579c-1537-4749-b82a-3dc973f77966 None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:14.671243 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a789579c-1537-4749-b82a-3dc973f77966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:14.817306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 887/1769] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Sep 8 16:08:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 08 16:08:14.917566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a70014-1817-4e02-8028-0149bbd46a2e admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:14.938397 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226758fa-919a-41a4-9e85-50e231d50b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:14.939070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226758fa-919a-41a4-9e85-50e231d50b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:14.939459 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226758fa-919a-41a4-9e85-50e231d50b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:14.973579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226758fa-919a-41a4-9e85-50e231d50b8c admin admin] PATH_INFO: `/v3/users/feb74b3c90ad4904a8dd3b05359908dc` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:14.981485 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226758fa-919a-41a4-9e85-50e231d50b8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb74b3c90ad4904a8dd3b05359908dc)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:15.005541 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226758fa-919a-41a4-9e85-50e231d50b8c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:15.131142 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-226758fa-919a-41a4-9e85-50e231d50b8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb74b3c90ad4904a8dd3b05359908dc'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:15.154851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789579c-1537-4749-b82a-3dc973f77966 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:15.156256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a789579c-1537-4749-b82a-3dc973f77966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:15.169595 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-226758fa-919a-41a4-9e85-50e231d50b8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb74b3c90ad4904a8dd3b05359908dc'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:15.170643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-226758fa-919a-41a4-9e85-50e231d50b8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb74b3c90ad4904a8dd3b05359908dc'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:15.171583 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-226758fa-919a-41a4-9e85-50e231d50b8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb74b3c90ad4904a8dd3b05359908dc'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:15.186245 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-226758fa-919a-41a4-9e85-50e231d50b8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb74b3c90ad4904a8dd3b05359908dc'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:15.250266 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 883/1770] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:14 2022] DELETE /identity/v3/users/feb74b3c90ad4904a8dd3b05359908dc => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:15.440758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789579c-1537-4749-b82a-3dc973f77966 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:15.477271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:15.582378 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749fdda-766b-4618-a119-e9a9983f3f49 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:15.584277 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749fdda-766b-4618-a119-e9a9983f3f49 service swift] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:15.595284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749fdda-766b-4618-a119-e9a9983f3f49 service swift] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:15.596436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749fdda-766b-4618-a119-e9a9983f3f49 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:15.601624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c749fdda-766b-4618-a119-e9a9983f3f49 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:15.855298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749fdda-766b-4618-a119-e9a9983f3f49 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:15.877720 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749fdda-766b-4618-a119-e9a9983f3f49 service swift] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:15.924812 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 888/1771] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:08:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 770 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:08:16.382255 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226758fa-919a-41a4-9e85-50e231d50b8c admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:16.384413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-226758fa-919a-41a4-9e85-50e231d50b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:16.481773 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:16.556262 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749fdda-766b-4618-a119-e9a9983f3f49 service swift] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:16.558076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c749fdda-766b-4618-a119-e9a9983f3f49 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:16.701172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226758fa-919a-41a4-9e85-50e231d50b8c admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:16.728503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb01f7-c865-417d-8372-f16b428f17d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:16.741454 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb01f7-c865-417d-8372-f16b428f17d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:16.747977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb01f7-c865-417d-8372-f16b428f17d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:16.748316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb01f7-c865-417d-8372-f16b428f17d0 admin admin] PATH_INFO: `/v3/users/83980d1e62ff463fae278ddd760c59ce` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:16.751319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb01f7-c865-417d-8372-f16b428f17d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83980d1e62ff463fae278ddd760c59ce)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:16.757751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb01f7-c865-417d-8372-f16b428f17d0 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:16.786473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749fdda-766b-4618-a119-e9a9983f3f49 service swift] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:16.800328 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1493dc-5a89-46b0-9230-3e1ca683c8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:16.803633 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1493dc-5a89-46b0-9230-3e1ca683c8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:16.805083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1493dc-5a89-46b0-9230-3e1ca683c8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:16.816903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1493dc-5a89-46b0-9230-3e1ca683c8d1 admin admin] PATH_INFO: `/v3/projects/884490340a3c402d992fe5fb1aa355a2` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:16.824297 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1493dc-5a89-46b0-9230-3e1ca683c8d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=884490340a3c402d992fe5fb1aa355a2)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:16.830504 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1493dc-5a89-46b0-9230-3e1ca683c8d1 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:16.944993 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-48fb01f7-c865-417d-8372-f16b428f17d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83980d1e62ff463fae278ddd760c59ce'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:16.978875 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-fc1493dc-5a89-46b0-9230-3e1ca683c8d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '884490340a3c402d992fe5fb1aa355a2', 'request_id': 'req-fc1493dc-5a89-46b0-9230-3e1ca683c8d1'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:16.998608 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-48fb01f7-c865-417d-8372-f16b428f17d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83980d1e62ff463fae278ddd760c59ce'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:17.000320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-fc1493dc-5a89-46b0-9230-3e1ca683c8d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '884490340a3c402d992fe5fb1aa355a2', 'request_id': 'req-fc1493dc-5a89-46b0-9230-3e1ca683c8d1'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:17.006400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-48fb01f7-c865-417d-8372-f16b428f17d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83980d1e62ff463fae278ddd760c59ce'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:17.008760 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-48fb01f7-c865-417d-8372-f16b428f17d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83980d1e62ff463fae278ddd760c59ce'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:17.014597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-fc1493dc-5a89-46b0-9230-3e1ca683c8d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '884490340a3c402d992fe5fb1aa355a2', 'request_id': 'req-fc1493dc-5a89-46b0-9230-3e1ca683c8d1'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:17.023835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:17.053594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-48fb01f7-c865-417d-8372-f16b428f17d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83980d1e62ff463fae278ddd760c59ce'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:17.065983 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-fc1493dc-5a89-46b0-9230-3e1ca683c8d1 admin admin] The token cache is being invalidate because project 884490340a3c402d992fe5fb1aa355a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:17.066277 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-fc1493dc-5a89-46b0-9230-3e1ca683c8d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:17.079178 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-fc1493dc-5a89-46b0-9230-3e1ca683c8d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:17.082129 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 889/1772] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:16 2022] DELETE /identity/v3/projects/884490340a3c402d992fe5fb1aa355a2 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:17.088270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eedad-6a48-40f8-bf4b-47ee720aa198 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:17.088528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eedad-6a48-40f8-bf4b-47ee720aa198 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:17.088755 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759eedad-6a48-40f8-bf4b-47ee720aa198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:17.128198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 884/1773] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:16 2022] DELETE /identity/v3/users/83980d1e62ff463fae278ddd760c59ce => generated 0 bytes in 746 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:17.142577 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad56314-1d66-44b5-be6a-c91bf5ef91c7 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:17.147985 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad56314-1d66-44b5-be6a-c91bf5ef91c7 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:17.150447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad56314-1d66-44b5-be6a-c91bf5ef91c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:17.155999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-759eedad-6a48-40f8-bf4b-47ee720aa198 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:17.157305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-759eedad-6a48-40f8-bf4b-47ee720aa198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:17.264754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-aad56314-1d66-44b5-be6a-c91bf5ef91c7 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:17.267340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-aad56314-1d66-44b5-be6a-c91bf5ef91c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:17.407132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 890/1774] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:17.416446 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723db4df-c2a7-424c-8030-104262f13592 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:17.417001 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723db4df-c2a7-424c-8030-104262f13592 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:17.417493 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723db4df-c2a7-424c-8030-104262f13592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:17.492569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-723db4df-c2a7-424c-8030-104262f13592 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:17.494525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-723db4df-c2a7-424c-8030-104262f13592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:17.517131 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 891/1775] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:17.522108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723db4df-c2a7-424c-8030-104262f13592 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:17.548052 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723db4df-c2a7-424c-8030-104262f13592 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:17.563380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66587449-619a-4344-837f-7dd7ccc5c1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:17.564497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66587449-619a-4344-837f-7dd7ccc5c1ff admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:17.564993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66587449-619a-4344-837f-7dd7ccc5c1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:17.565467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66587449-619a-4344-837f-7dd7ccc5c1ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:17.566338 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66587449-619a-4344-837f-7dd7ccc5c1ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:17.566854 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66587449-619a-4344-837f-7dd7ccc5c1ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:17.568845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66587449-619a-4344-837f-7dd7ccc5c1ff admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:17.591715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 885/1776] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:17.600726 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 892/1777] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:08:17.606113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad56314-1d66-44b5-be6a-c91bf5ef91c7 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:17.692061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad56314-1d66-44b5-be6a-c91bf5ef91c7 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:17.714257 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898ef78-20cb-49dd-942b-fd67523f96a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:17.718692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898ef78-20cb-49dd-942b-fd67523f96a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:17.720989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898ef78-20cb-49dd-942b-fd67523f96a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:17.723386 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8898ef78-20cb-49dd-942b-fd67523f96a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:17.724482 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898ef78-20cb-49dd-942b-fd67523f96a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:17.730404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8898ef78-20cb-49dd-942b-fd67523f96a0 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:17.830358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66587449-619a-4344-837f-7dd7ccc5c1ff admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:17.832400 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-66587449-619a-4344-837f-7dd7ccc5c1ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:17.836641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 886/1778] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:17 2022] POST /identity/v3/projects => generated 363 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:17.837630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898ef78-20cb-49dd-942b-fd67523f96a0 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:17.968760 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:18.057814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66587449-619a-4344-837f-7dd7ccc5c1ff admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:18.156259 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8898ef78-20cb-49dd-942b-fd67523f96a0 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:18.177327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3e120-3b70-41cb-ab4f-5e981a15d2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:18.178217 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3e120-3b70-41cb-ab4f-5e981a15d2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:18.178674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3e120-3b70-41cb-ab4f-5e981a15d2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:18.179100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3e120-3b70-41cb-ab4f-5e981a15d2b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:18.179975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb3e120-3b70-41cb-ab4f-5e981a15d2b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:18.182098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb3e120-3b70-41cb-ab4f-5e981a15d2b8 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:18.232119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 887/1779] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:17 2022] POST /identity/v3/users => generated 282 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:18.237034 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3e120-3b70-41cb-ab4f-5e981a15d2b8 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:18.265250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3e120-3b70-41cb-ab4f-5e981a15d2b8 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:18.270047 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10b800e-051c-4417-89ec-1d6f1d8b968e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:18.270894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b800e-051c-4417-89ec-1d6f1d8b968e service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:18.271352 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b800e-051c-4417-89ec-1d6f1d8b968e service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:18.271989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10b800e-051c-4417-89ec-1d6f1d8b968e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:18.284468 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb78d7-2a03-4340-a91c-e9fe95f4c62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:18.285479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb78d7-2a03-4340-a91c-e9fe95f4c62d admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:18.285479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb78d7-2a03-4340-a91c-e9fe95f4c62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:18.285711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb78d7-2a03-4340-a91c-e9fe95f4c62d admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:18.287752 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb78d7-2a03-4340-a91c-e9fe95f4c62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:18.289387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb78d7-2a03-4340-a91c-e9fe95f4c62d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:18.322850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 888/1780] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:08:18.327599 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb78d7-2a03-4340-a91c-e9fe95f4c62d admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:18.354942 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb78d7-2a03-4340-a91c-e9fe95f4c62d admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:18.368575 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b800e-051c-4417-89ec-1d6f1d8b968e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:18.375859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10b800e-051c-4417-89ec-1d6f1d8b968e service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:18.377738 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d103a9-696e-40ad-b257-83bb8bab1a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:18.378676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d103a9-696e-40ad-b257-83bb8bab1a67 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:18.379101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d103a9-696e-40ad-b257-83bb8bab1a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:18.379554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d103a9-696e-40ad-b257-83bb8bab1a67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:18.382105 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d103a9-696e-40ad-b257-83bb8bab1a67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:18.385129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d103a9-696e-40ad-b257-83bb8bab1a67 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:18.420654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 889/1781] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:18.425499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d103a9-696e-40ad-b257-83bb8bab1a67 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:18.447143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 893/1782] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:18.466373 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d103a9-696e-40ad-b257-83bb8bab1a67 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:18.501391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904a73c-ac5c-4e13-bb38-c8ad4c39c852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:18.502547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904a73c-ac5c-4e13-bb38-c8ad4c39c852 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:18.503080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904a73c-ac5c-4e13-bb38-c8ad4c39c852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:18.503521 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904a73c-ac5c-4e13-bb38-c8ad4c39c852 admin admin] PATH_INFO: `/v3/projects/b069c83932a94a9c84c3a7a58465fc7c/users/ce41d643c0054520968378c9a1b9a69f/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:18.583399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904a73c-ac5c-4e13-bb38-c8ad4c39c852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b069c83932a94a9c84c3a7a58465fc7c, user_id=ce41d643c0054520968378c9a1b9a69f, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:18.586170 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f904a73c-ac5c-4e13-bb38-c8ad4c39c852 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:18.628714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 890/1783] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:18 2022] PUT /identity/v3/projects/b069c83932a94a9c84c3a7a58465fc7c/users/ce41d643c0054520968378c9a1b9a69f/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:18.640141 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06c03a-0216-427a-bf87-926e01c0e991 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:18.640723 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06c03a-0216-427a-bf87-926e01c0e991 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:18.641276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06c03a-0216-427a-bf87-926e01c0e991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:18.727029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-dc06c03a-0216-427a-bf87-926e01c0e991 None None] MFA Rules not processed for user `ce41d643c0054520968378c9a1b9a69f`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:18.729425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-dc06c03a-0216-427a-bf87-926e01c0e991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:18.862543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:18.985552 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904a73c-ac5c-4e13-bb38-c8ad4c39c852 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:18.998377 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 894/1784] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:19.041548 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06c03a-0216-427a-bf87-926e01c0e991 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:19.043699 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-dc06c03a-0216-427a-bf87-926e01c0e991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:19.218068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904a73c-ac5c-4e13-bb38-c8ad4c39c852 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:19.228229 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb642d8d-fc3f-46d1-ae17-6eae5c7ee8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:19.241404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb642d8d-fc3f-46d1-ae17-6eae5c7ee8a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:19.241404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb642d8d-fc3f-46d1-ae17-6eae5c7ee8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:19.241404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb642d8d-fc3f-46d1-ae17-6eae5c7ee8a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:19.241404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb642d8d-fc3f-46d1-ae17-6eae5c7ee8a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:19.242907 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb642d8d-fc3f-46d1-ae17-6eae5c7ee8a4 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:19.265376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06c03a-0216-427a-bf87-926e01c0e991 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:19.277920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8264202-9de4-43c4-ac93-4c51e3ad022f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:19.278695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8264202-9de4-43c4-ac93-4c51e3ad022f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:19.279075 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8264202-9de4-43c4-ac93-4c51e3ad022f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:19.279874 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8264202-9de4-43c4-ac93-4c51e3ad022f admin admin] PATH_INFO: `/v3/users/1bc4135eb1c74811bf31064218941bb2` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:19.283752 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8264202-9de4-43c4-ac93-4c51e3ad022f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bc4135eb1c74811bf31064218941bb2)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:19.289647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8264202-9de4-43c4-ac93-4c51e3ad022f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:19.328227 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 891/1785] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:18 2022] POST /identity/v3/projects => generated 363 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:19.329775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023c8f8-87ad-47b8-b17b-512b636e8f27 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:19.330160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023c8f8-87ad-47b8-b17b-512b636e8f27 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:19.330535 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023c8f8-87ad-47b8-b17b-512b636e8f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:19.366217 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-d023c8f8-87ad-47b8-b17b-512b636e8f27 None None] MFA Rules not processed for user `3aae65f9ec394f97aeafad43932dc49c`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:19.368883 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d023c8f8-87ad-47b8-b17b-512b636e8f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:19.429540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c8264202-9de4-43c4-ac93-4c51e3ad022f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bc4135eb1c74811bf31064218941bb2'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:19.449849 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c8264202-9de4-43c4-ac93-4c51e3ad022f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bc4135eb1c74811bf31064218941bb2'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:19.451157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c8264202-9de4-43c4-ac93-4c51e3ad022f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bc4135eb1c74811bf31064218941bb2'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:19.452078 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c8264202-9de4-43c4-ac93-4c51e3ad022f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bc4135eb1c74811bf31064218941bb2'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:19.489182 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c8264202-9de4-43c4-ac93-4c51e3ad022f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bc4135eb1c74811bf31064218941bb2'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:19.492269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:19.513893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 892/1786] 10.222.0.20 () {58 vars in 1015 bytes} [Thu Sep 8 16:08:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:19.515548 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023c8f8-87ad-47b8-b17b-512b636e8f27 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:19.516213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 895/1787] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:19 2022] DELETE /identity/v3/users/1bc4135eb1c74811bf31064218941bb2 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:19.517671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d023c8f8-87ad-47b8-b17b-512b636e8f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:19.524932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2387e-411f-42f6-80c7-97699a8dea65 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:19.525414 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2387e-411f-42f6-80c7-97699a8dea65 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:19.525696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2387e-411f-42f6-80c7-97699a8dea65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:19.581020 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-1bb2387e-411f-42f6-80c7-97699a8dea65 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:19.582293 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-1bb2387e-411f-42f6-80c7-97699a8dea65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:19.659269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023c8f8-87ad-47b8-b17b-512b636e8f27 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:19.699277 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:19.742510 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 896/1788] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:19.745815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2387e-411f-42f6-80c7-97699a8dea65 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:19.747014 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-1bb2387e-411f-42f6-80c7-97699a8dea65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:19.814140 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe10c2-64ab-450b-8ab8-609d6072ce4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:19.816979 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe10c2-64ab-450b-8ab8-609d6072ce4c admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:19.817234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe10c2-64ab-450b-8ab8-609d6072ce4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:19.817967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe10c2-64ab-450b-8ab8-609d6072ce4c admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:19.818822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe10c2-64ab-450b-8ab8-609d6072ce4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:19.822620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffe10c2-64ab-450b-8ab8-609d6072ce4c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:19.898564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 893/1789] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:19 2022] POST /identity/v3/users => generated 288 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:19.901727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe10c2-64ab-450b-8ab8-609d6072ce4c admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:19.955125 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe10c2-64ab-450b-8ab8-609d6072ce4c admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:19.972408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2387e-411f-42f6-80c7-97699a8dea65 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:19.982125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be080d-f42d-4834-b99b-7cd88c64ed0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:19.982772 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be080d-f42d-4834-b99b-7cd88c64ed0a service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:19.983046 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be080d-f42d-4834-b99b-7cd88c64ed0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:19.983317 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be080d-f42d-4834-b99b-7cd88c64ed0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:19.991619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75dfd14-c9c4-4da7-8920-ffffc520ffbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:19.992265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dfd14-c9c4-4da7-8920-ffffc520ffbd service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:19.992594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dfd14-c9c4-4da7-8920-ffffc520ffbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:19.992808 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75dfd14-c9c4-4da7-8920-ffffc520ffbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:19.999907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be080d-f42d-4834-b99b-7cd88c64ed0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:20.001393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be080d-f42d-4834-b99b-7cd88c64ed0a service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:20.024007 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75dfd14-c9c4-4da7-8920-ffffc520ffbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:20.025802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75dfd14-c9c4-4da7-8920-ffffc520ffbd service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:20.060024 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 897/1790] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:20.062055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be080d-f42d-4834-b99b-7cd88c64ed0a service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:20.084330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 894/1791] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:20.095169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be080d-f42d-4834-b99b-7cd88c64ed0a service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:20.106915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a801bc-201f-4da4-b2bb-e4c22e8b5af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:20.107712 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a801bc-201f-4da4-b2bb-e4c22e8b5af4 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:20.108080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a801bc-201f-4da4-b2bb-e4c22e8b5af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:20.108359 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a801bc-201f-4da4-b2bb-e4c22e8b5af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:20.109086 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a801bc-201f-4da4-b2bb-e4c22e8b5af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:20.112254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a801bc-201f-4da4-b2bb-e4c22e8b5af4 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:20.146909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75dfd14-c9c4-4da7-8920-ffffc520ffbd service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:20.147565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 898/1792] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:20.175796 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75dfd14-c9c4-4da7-8920-ffffc520ffbd service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:20.216746 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2cc52-f9cf-468b-be5c-320b4fab6195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:20.217484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2cc52-f9cf-468b-be5c-320b4fab6195 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:20.217762 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2cc52-f9cf-468b-be5c-320b4fab6195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:20.218028 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2cc52-f9cf-468b-be5c-320b4fab6195 admin admin] PATH_INFO: `/v3/projects/ded4735dd38e4a7882360256d1b2c7fc/users/8f97a8ea0a2740ba8f095759a42530ed/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:20.303839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2cc52-f9cf-468b-be5c-320b4fab6195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded4735dd38e4a7882360256d1b2c7fc, user_id=8f97a8ea0a2740ba8f095759a42530ed, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:20.306817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2cc52-f9cf-468b-be5c-320b4fab6195 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:20.407021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 895/1793] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:20 2022] PUT /identity/v3/projects/ded4735dd38e4a7882360256d1b2c7fc/users/8f97a8ea0a2740ba8f095759a42530ed/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:20.416718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a801bc-201f-4da4-b2bb-e4c22e8b5af4 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:20.447099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a801bc-201f-4da4-b2bb-e4c22e8b5af4 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:20.603364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfa9bb-6259-4677-9447-7119e4bd9577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:20.604157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfa9bb-6259-4677-9447-7119e4bd9577 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:20.604612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfa9bb-6259-4677-9447-7119e4bd9577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:20.605002 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfa9bb-6259-4677-9447-7119e4bd9577 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:20.605794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dfa9bb-6259-4677-9447-7119e4bd9577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:20.607575 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dfa9bb-6259-4677-9447-7119e4bd9577 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:20.633275 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 899/1794] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:20.634906 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2cc52-f9cf-468b-be5c-320b4fab6195 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:20.667677 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2cc52-f9cf-468b-be5c-320b4fab6195 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:20.686339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a1ac7-8bfb-42e6-a514-aaa8c256b31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:20.688464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a1ac7-8bfb-42e6-a514-aaa8c256b31c admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:20.689045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a1ac7-8bfb-42e6-a514-aaa8c256b31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:20.690151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a1ac7-8bfb-42e6-a514-aaa8c256b31c admin admin] PATH_INFO: `/v3/projects/ded4735dd38e4a7882360256d1b2c7fc/users/8f97a8ea0a2740ba8f095759a42530ed/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:20.695582 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a1ac7-8bfb-42e6-a514-aaa8c256b31c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded4735dd38e4a7882360256d1b2c7fc, user_id=8f97a8ea0a2740ba8f095759a42530ed, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:20.703037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a1ac7-8bfb-42e6-a514-aaa8c256b31c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:20.733806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 896/1795] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:20 2022] PUT /identity/v3/projects/ded4735dd38e4a7882360256d1b2c7fc/users/8f97a8ea0a2740ba8f095759a42530ed/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:20.751763 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2a5f0-ba5e-4c10-9dbe-6dfea3174d9f None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:20.752381 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2a5f0-ba5e-4c10-9dbe-6dfea3174d9f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:20.752871 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2a5f0-ba5e-4c10-9dbe-6dfea3174d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:20.832611 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-6eb2a5f0-ba5e-4c10-9dbe-6dfea3174d9f None None] MFA Rules not processed for user `8f97a8ea0a2740ba8f095759a42530ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:20.859479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6eb2a5f0-ba5e-4c10-9dbe-6dfea3174d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:21.091677 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:21.130492 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 900/1796] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:21.167191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a1ac7-8bfb-42e6-a514-aaa8c256b31c admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:21.173225 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-154a1ac7-8bfb-42e6-a514-aaa8c256b31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:21.174821 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb2a5f0-ba5e-4c10-9dbe-6dfea3174d9f None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:21.176342 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6eb2a5f0-ba5e-4c10-9dbe-6dfea3174d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:21.335584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:21.361620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a1ac7-8bfb-42e6-a514-aaa8c256b31c admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:21.370693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb2a5f0-ba5e-4c10-9dbe-6dfea3174d9f None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:21.377816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fff0b-af3b-4375-91bb-b38859501343 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:21.378472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fff0b-af3b-4375-91bb-b38859501343 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:21.378735 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fff0b-af3b-4375-91bb-b38859501343 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:21.379008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fff0b-af3b-4375-91bb-b38859501343 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:21.384799 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7b32f-addc-4671-bd8d-4705d622958f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:21.385599 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7b32f-addc-4671-bd8d-4705d622958f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:21.386006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7b32f-addc-4671-bd8d-4705d622958f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:21.386375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7b32f-addc-4671-bd8d-4705d622958f admin admin] PATH_INFO: `/v3/projects/a829c94f7acb470c8908b01460e9f070` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:21.387785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7b32f-addc-4671-bd8d-4705d622958f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a829c94f7acb470c8908b01460e9f070)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:21.390064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7b32f-addc-4671-bd8d-4705d622958f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:21.395854 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71fff0b-af3b-4375-91bb-b38859501343 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:21.398203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71fff0b-af3b-4375-91bb-b38859501343 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:21.476949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 901/1797] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:08:21.582038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-82c7b32f-addc-4671-bd8d-4705d622958f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a829c94f7acb470c8908b01460e9f070', 'request_id': 'req-82c7b32f-addc-4671-bd8d-4705d622958f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:21.606223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-82c7b32f-addc-4671-bd8d-4705d622958f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a829c94f7acb470c8908b01460e9f070', 'request_id': 'req-82c7b32f-addc-4671-bd8d-4705d622958f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:21.621609 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-82c7b32f-addc-4671-bd8d-4705d622958f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a829c94f7acb470c8908b01460e9f070', 'request_id': 'req-82c7b32f-addc-4671-bd8d-4705d622958f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:21.657725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fff0b-af3b-4375-91bb-b38859501343 service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:21.664426 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-82c7b32f-addc-4671-bd8d-4705d622958f admin admin] The token cache is being invalidate because project a829c94f7acb470c8908b01460e9f070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:21.664789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-82c7b32f-addc-4671-bd8d-4705d622958f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:21.674941 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-82c7b32f-addc-4671-bd8d-4705d622958f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:21.678758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fff0b-af3b-4375-91bb-b38859501343 service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:21.685167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 897/1798] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:21 2022] DELETE /identity/v3/projects/a829c94f7acb470c8908b01460e9f070 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:21.700655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7b32f-addc-4671-bd8d-4705d622958f admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:21.710794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-82c7b32f-addc-4671-bd8d-4705d622958f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:21.900193 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b14ac-46a1-4ac8-a2a2-8d0dcfb7c31c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:21.911083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b14ac-46a1-4ac8-a2a2-8d0dcfb7c31c service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:21.911796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b14ac-46a1-4ac8-a2a2-8d0dcfb7c31c service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:21.914393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7b32f-addc-4671-bd8d-4705d622958f admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:21.914783 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b14ac-46a1-4ac8-a2a2-8d0dcfb7c31c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:21.924865 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b35b14ac-46a1-4ac8-a2a2-8d0dcfb7c31c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:21.931971 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6370eb8-5187-476b-a70f-932b61817e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:21.932641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6370eb8-5187-476b-a70f-932b61817e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:21.933029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6370eb8-5187-476b-a70f-932b61817e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:21.933304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6370eb8-5187-476b-a70f-932b61817e48 admin admin] PATH_INFO: `/v3/users/eb84d447806249a48f85a563c3183aee` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:21.934963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6370eb8-5187-476b-a70f-932b61817e48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb84d447806249a48f85a563c3183aee)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:21.937070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6370eb8-5187-476b-a70f-932b61817e48 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:22.091482 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e6370eb8-5187-476b-a70f-932b61817e48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb84d447806249a48f85a563c3183aee'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:22.118804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e6370eb8-5187-476b-a70f-932b61817e48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb84d447806249a48f85a563c3183aee'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:22.119798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e6370eb8-5187-476b-a70f-932b61817e48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb84d447806249a48f85a563c3183aee'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:22.120713 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e6370eb8-5187-476b-a70f-932b61817e48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb84d447806249a48f85a563c3183aee'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:22.132651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e6370eb8-5187-476b-a70f-932b61817e48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb84d447806249a48f85a563c3183aee'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:22.174536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:22.184402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 898/1799] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:21 2022] DELETE /identity/v3/users/eb84d447806249a48f85a563c3183aee => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:22.238334 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b14ac-46a1-4ac8-a2a2-8d0dcfb7c31c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:22.239875 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b14ac-46a1-4ac8-a2a2-8d0dcfb7c31c service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:22.285861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 902/1800] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:08:23.695178 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6370eb8-5187-476b-a70f-932b61817e48 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:23.697666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e6370eb8-5187-476b-a70f-932b61817e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:24.008596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6370eb8-5187-476b-a70f-932b61817e48 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:24.038118 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f01c6c-73d1-4dca-92b1-4f1038455d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:24.040219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f01c6c-73d1-4dca-92b1-4f1038455d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:24.040800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f01c6c-73d1-4dca-92b1-4f1038455d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:24.043153 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f01c6c-73d1-4dca-92b1-4f1038455d3a admin admin] PATH_INFO: `/v3/projects/badc184402544a39868f96c9dd9c7f87` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:24.050417 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f01c6c-73d1-4dca-92b1-4f1038455d3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=badc184402544a39868f96c9dd9c7f87)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:24.055376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f01c6c-73d1-4dca-92b1-4f1038455d3a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:24.316497 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-07f01c6c-73d1-4dca-92b1-4f1038455d3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'badc184402544a39868f96c9dd9c7f87', 'request_id': 'req-07f01c6c-73d1-4dca-92b1-4f1038455d3a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:24.336029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-07f01c6c-73d1-4dca-92b1-4f1038455d3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'badc184402544a39868f96c9dd9c7f87', 'request_id': 'req-07f01c6c-73d1-4dca-92b1-4f1038455d3a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:24.346829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-07f01c6c-73d1-4dca-92b1-4f1038455d3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'badc184402544a39868f96c9dd9c7f87', 'request_id': 'req-07f01c6c-73d1-4dca-92b1-4f1038455d3a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:24.372050 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-07f01c6c-73d1-4dca-92b1-4f1038455d3a admin admin] The token cache is being invalidate because project badc184402544a39868f96c9dd9c7f87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:24.372346 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-07f01c6c-73d1-4dca-92b1-4f1038455d3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:24.374583 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-07f01c6c-73d1-4dca-92b1-4f1038455d3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:24.378016 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 899/1801] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:23 2022] DELETE /identity/v3/projects/badc184402544a39868f96c9dd9c7f87 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:24.382251 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b14ac-46a1-4ac8-a2a2-8d0dcfb7c31c service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:24.384062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b35b14ac-46a1-4ac8-a2a2-8d0dcfb7c31c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:24.410852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b14ac-46a1-4ac8-a2a2-8d0dcfb7c31c service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:24.562916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:24.600029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3263fdf-5822-4971-b2e3-7d2c71ec67e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:24.601012 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3263fdf-5822-4971-b2e3-7d2c71ec67e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:24.603972 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3263fdf-5822-4971-b2e3-7d2c71ec67e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:24.605208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3263fdf-5822-4971-b2e3-7d2c71ec67e4 admin admin] PATH_INFO: `/v3/users/0deb38c37fca47afab4cb5686ad9d4cf` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:24.611067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3263fdf-5822-4971-b2e3-7d2c71ec67e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0deb38c37fca47afab4cb5686ad9d4cf)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:24.613200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3263fdf-5822-4971-b2e3-7d2c71ec67e4 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:24.843506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a3263fdf-5822-4971-b2e3-7d2c71ec67e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0deb38c37fca47afab4cb5686ad9d4cf'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:24.857199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a3263fdf-5822-4971-b2e3-7d2c71ec67e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0deb38c37fca47afab4cb5686ad9d4cf'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:24.858739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a3263fdf-5822-4971-b2e3-7d2c71ec67e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0deb38c37fca47afab4cb5686ad9d4cf'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:24.859662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a3263fdf-5822-4971-b2e3-7d2c71ec67e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0deb38c37fca47afab4cb5686ad9d4cf'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:24.892679 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a3263fdf-5822-4971-b2e3-7d2c71ec67e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0deb38c37fca47afab4cb5686ad9d4cf'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:24.948942 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 903/1802] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:24 2022] DELETE /identity/v3/users/0deb38c37fca47afab4cb5686ad9d4cf => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:25.789127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f01c6c-73d1-4dca-92b1-4f1038455d3a admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:25.790811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-07f01c6c-73d1-4dca-92b1-4f1038455d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:25.814047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f01c6c-73d1-4dca-92b1-4f1038455d3a admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:25.952836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81831598-3058-497f-84e6-c9effe1754e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:25.953600 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81831598-3058-497f-84e6-c9effe1754e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:25.953910 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81831598-3058-497f-84e6-c9effe1754e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:25.954186 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81831598-3058-497f-84e6-c9effe1754e0 admin admin] PATH_INFO: `/v3/projects/45a000f6a7cf4acf80da045a99630f91` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:25.955548 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81831598-3058-497f-84e6-c9effe1754e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45a000f6a7cf4acf80da045a99630f91)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:25.957502 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81831598-3058-497f-84e6-c9effe1754e0 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:26.032376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:26.067255 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-81831598-3058-497f-84e6-c9effe1754e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45a000f6a7cf4acf80da045a99630f91', 'request_id': 'req-81831598-3058-497f-84e6-c9effe1754e0'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:26.084698 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-81831598-3058-497f-84e6-c9effe1754e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45a000f6a7cf4acf80da045a99630f91', 'request_id': 'req-81831598-3058-497f-84e6-c9effe1754e0'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:26.100075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-81831598-3058-497f-84e6-c9effe1754e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45a000f6a7cf4acf80da045a99630f91', 'request_id': 'req-81831598-3058-497f-84e6-c9effe1754e0'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:26.135380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-81831598-3058-497f-84e6-c9effe1754e0 admin admin] The token cache is being invalidate because project 45a000f6a7cf4acf80da045a99630f91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:26.135700 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-81831598-3058-497f-84e6-c9effe1754e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:26.137871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-81831598-3058-497f-84e6-c9effe1754e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:26.144358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 900/1803] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:25 2022] DELETE /identity/v3/projects/45a000f6a7cf4acf80da045a99630f91 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:26.148992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e13e0-5057-4175-99ea-7ef9330a7c25 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:26.149514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e13e0-5057-4175-99ea-7ef9330a7c25 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:26.149514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e13e0-5057-4175-99ea-7ef9330a7c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:26.198112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-2f5e13e0-5057-4175-99ea-7ef9330a7c25 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:26.202240 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2f5e13e0-5057-4175-99ea-7ef9330a7c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:26.405668 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81831598-3058-497f-84e6-c9effe1754e0 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:26.407212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-81831598-3058-497f-84e6-c9effe1754e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:26.451876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 904/1804] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:26.453588 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cc7b9-ee3b-4c32-a88d-93f3da4ec7f6 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:26.454019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cc7b9-ee3b-4c32-a88d-93f3da4ec7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:26.454519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10cc7b9-ee3b-4c32-a88d-93f3da4ec7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:26.512265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-a10cc7b9-ee3b-4c32-a88d-93f3da4ec7f6 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:26.514044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a10cc7b9-ee3b-4c32-a88d-93f3da4ec7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:26.544027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 905/1805] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:26.552216 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10cc7b9-ee3b-4c32-a88d-93f3da4ec7f6 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:26.578715 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10cc7b9-ee3b-4c32-a88d-93f3da4ec7f6 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:26.581561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81831598-3058-497f-84e6-c9effe1754e0 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:26.587681 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787edb3-573f-435f-950b-db21b006352b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:26.588484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787edb3-573f-435f-950b-db21b006352b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:26.588990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787edb3-573f-435f-950b-db21b006352b admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:26.589495 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787edb3-573f-435f-950b-db21b006352b admin admin] PATH_INFO: `/v3/users/ce41d643c0054520968378c9a1b9a69f` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:26.591219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787edb3-573f-435f-950b-db21b006352b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce41d643c0054520968378c9a1b9a69f)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:26.600708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8828737-28e8-4700-a699-460505143a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:26.601375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787edb3-573f-435f-950b-db21b006352b admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:26.602173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8828737-28e8-4700-a699-460505143a5b admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:26.604408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8828737-28e8-4700-a699-460505143a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:26.606292 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8828737-28e8-4700-a699-460505143a5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:26.610488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8828737-28e8-4700-a699-460505143a5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:26.612624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8828737-28e8-4700-a699-460505143a5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:26.642206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8828737-28e8-4700-a699-460505143a5b admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:26.705109 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 906/1806] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:08:26.710166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8828737-28e8-4700-a699-460505143a5b admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:26.733350 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8828737-28e8-4700-a699-460505143a5b admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:26.742058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2787edb3-573f-435f-950b-db21b006352b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce41d643c0054520968378c9a1b9a69f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:26.759155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426d6fc-aff9-4b3f-8b7e-70d640c16338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:26.760484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426d6fc-aff9-4b3f-8b7e-70d640c16338 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:26.761210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426d6fc-aff9-4b3f-8b7e-70d640c16338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:26.761495 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4426d6fc-aff9-4b3f-8b7e-70d640c16338 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:26.762242 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426d6fc-aff9-4b3f-8b7e-70d640c16338 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:26.765136 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4426d6fc-aff9-4b3f-8b7e-70d640c16338 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:26.768693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2787edb3-573f-435f-950b-db21b006352b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce41d643c0054520968378c9a1b9a69f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:26.769611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2787edb3-573f-435f-950b-db21b006352b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce41d643c0054520968378c9a1b9a69f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:26.770432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2787edb3-573f-435f-950b-db21b006352b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce41d643c0054520968378c9a1b9a69f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:26.789622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2787edb3-573f-435f-950b-db21b006352b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce41d643c0054520968378c9a1b9a69f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:26.831564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 907/1807] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:26 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:26.832711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426d6fc-aff9-4b3f-8b7e-70d640c16338 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:26.837277 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 901/1808] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:26 2022] DELETE /identity/v3/users/ce41d643c0054520968378c9a1b9a69f => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:26.838172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-4426d6fc-aff9-4b3f-8b7e-70d640c16338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:26.839329 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69bf47-a679-465d-98c4-5f4678af3ca5 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:26.839739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69bf47-a679-465d-98c4-5f4678af3ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:26.840145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69bf47-a679-465d-98c4-5f4678af3ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:26.887345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-9e69bf47-a679-465d-98c4-5f4678af3ca5 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:26.888959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9e69bf47-a679-465d-98c4-5f4678af3ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:26.896932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:26.961569 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:27.026399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4426d6fc-aff9-4b3f-8b7e-70d640c16338 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:27.027113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 902/1809] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:27.051377 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32acdfd-f136-457d-aeeb-d25965c3c05c None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:27.051794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32acdfd-f136-457d-aeeb-d25965c3c05c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:27.052123 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32acdfd-f136-457d-aeeb-d25965c3c05c None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:27.053445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60794748-6e1d-4f34-9871-6c95372beedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:27.055354 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60794748-6e1d-4f34-9871-6c95372beedc admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:27.055577 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 903/1810] 10.222.0.27 () {54 vars in 921 bytes} [Thu Sep 8 16:08:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 16:08:27.056304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60794748-6e1d-4f34-9871-6c95372beedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:27.056785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60794748-6e1d-4f34-9871-6c95372beedc admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:27.057676 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60794748-6e1d-4f34-9871-6c95372beedc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:27.070966 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60794748-6e1d-4f34-9871-6c95372beedc admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:27.081403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c9165-0c84-4cfb-a53a-c5c1785e61d0 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:27.081775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c9165-0c84-4cfb-a53a-c5c1785e61d0 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:27.082002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c9165-0c84-4cfb-a53a-c5c1785e61d0 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:27.083855 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 904/1811] 10.222.0.27 () {54 vars in 921 bytes} [Thu Sep 8 16:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:08:27.122827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c9165-0c84-4cfb-a53a-c5c1785e61d0 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:27.124082 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 908/1812] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:26 2022] POST /identity/v3/users => generated 293 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:08:27.148916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60794748-6e1d-4f34-9871-6c95372beedc admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:27.151245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-60794748-6e1d-4f34-9871-6c95372beedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:27.155053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c9165-0c84-4cfb-a53a-c5c1785e61d0 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:27.173914 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4360a-4360-444e-95d5-6a5daaef6a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:27.174654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4360a-4360-444e-95d5-6a5daaef6a4d admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:27.174937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4360a-4360-444e-95d5-6a5daaef6a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:27.175257 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4360a-4360-444e-95d5-6a5daaef6a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:27.178313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4360a-4360-444e-95d5-6a5daaef6a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:27.181703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4360a-4360-444e-95d5-6a5daaef6a4d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:27.203579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 905/1813] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:08:27.205467 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfd089-73ac-441a-a341-7bccbf7e8ae8 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:27.206075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfd089-73ac-441a-a341-7bccbf7e8ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:27.206511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfd089-73ac-441a-a341-7bccbf7e8ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:27.238682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-b4bfd089-73ac-441a-a341-7bccbf7e8ae8 None None] MFA Rules not processed for user `6e9209e77ce44b128e25a8ddcf78f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:27.240265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b4bfd089-73ac-441a-a341-7bccbf7e8ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:27.313323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60794748-6e1d-4f34-9871-6c95372beedc admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:27.326011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60734955-0228-4468-b0cc-ede34d8f868d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:27.326832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60734955-0228-4468-b0cc-ede34d8f868d service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:27.328522 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60734955-0228-4468-b0cc-ede34d8f868d service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:27.328981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60734955-0228-4468-b0cc-ede34d8f868d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:27.356022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60734955-0228-4468-b0cc-ede34d8f868d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:27.358294 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60734955-0228-4468-b0cc-ede34d8f868d service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:27.388609 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 909/1814] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:08:27.393805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde8388-c721-4788-b909-be97a8513d64 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:27.395589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde8388-c721-4788-b909-be97a8513d64 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:27.396481 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dde8388-c721-4788-b909-be97a8513d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:27.451753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 906/1815] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Sep 8 16:08:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:27.457673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-4dde8388-c721-4788-b909-be97a8513d64 None None] MFA Rules not processed for user `6e9209e77ce44b128e25a8ddcf78f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:27.459098 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-4dde8388-c721-4788-b909-be97a8513d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:27.470082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bfd089-73ac-441a-a341-7bccbf7e8ae8 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:27.482638 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 910/1816] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Sep 8 16:08:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:27.487730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bfd089-73ac-441a-a341-7bccbf7e8ae8 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:27.495410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5c463-e56c-497b-a423-cb39deb88d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:27.496297 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5c463-e56c-497b-a423-cb39deb88d0b admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:27.496695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5c463-e56c-497b-a423-cb39deb88d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:27.497065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5c463-e56c-497b-a423-cb39deb88d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:27.498350 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5c463-e56c-497b-a423-cb39deb88d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:27.521251 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a5c463-e56c-497b-a423-cb39deb88d0b admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:27.557059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde8388-c721-4788-b909-be97a8513d64 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:27.561517 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-4dde8388-c721-4788-b909-be97a8513d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:27.565710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 907/1817] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:27.570303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5c463-e56c-497b-a423-cb39deb88d0b admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:27.597801 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a5c463-e56c-497b-a423-cb39deb88d0b admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:27.606073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dde8388-c721-4788-b909-be97a8513d64 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:27.614427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef71761-dbb0-4996-907b-acaea21a3ce0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:27.615169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef71761-dbb0-4996-907b-acaea21a3ce0 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:27.615598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef71761-dbb0-4996-907b-acaea21a3ce0 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:27.615935 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef71761-dbb0-4996-907b-acaea21a3ce0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:27.616101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c223c-ea7f-46f1-8dd5-0490dd39fa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:27.618088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c223c-ea7f-46f1-8dd5-0490dd39fa8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:27.633049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef71761-dbb0-4996-907b-acaea21a3ce0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:27.634753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c223c-ea7f-46f1-8dd5-0490dd39fa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:27.636240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439c223c-ea7f-46f1-8dd5-0490dd39fa8c admin admin] PATH_INFO: `/v3/projects/d26c35b041b54f609b9e2e4dff4cb0f7/users/53329b1a7ac54bdfa452d876d5c44fc1/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:27.637468 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef71761-dbb0-4996-907b-acaea21a3ce0 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:27.659564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 911/1818] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:27.699035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c223c-ea7f-46f1-8dd5-0490dd39fa8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d26c35b041b54f609b9e2e4dff4cb0f7, user_id=53329b1a7ac54bdfa452d876d5c44fc1, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:27.706170 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439c223c-ea7f-46f1-8dd5-0490dd39fa8c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:27.762185 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 908/1819] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:27 2022] PUT /identity/v3/projects/d26c35b041b54f609b9e2e4dff4cb0f7/users/53329b1a7ac54bdfa452d876d5c44fc1/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:27.766037 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a0356-faa6-428d-8c99-221f0ba9a9f6 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:27.766545 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a0356-faa6-428d-8c99-221f0ba9a9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:27.766965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a0356-faa6-428d-8c99-221f0ba9a9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:27.840052 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c223c-ea7f-46f1-8dd5-0490dd39fa8c admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:27.911766 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-016a0356-faa6-428d-8c99-221f0ba9a9f6 None None] MFA Rules not processed for user `53329b1a7ac54bdfa452d876d5c44fc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:27.918259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-016a0356-faa6-428d-8c99-221f0ba9a9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:28.170068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439c223c-ea7f-46f1-8dd5-0490dd39fa8c admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:28.181111 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecf365-3451-4c46-a508-775157cda6ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:28.183410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecf365-3451-4c46-a508-775157cda6ad service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:28.184220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecf365-3451-4c46-a508-775157cda6ad service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:28.184733 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ecf365-3451-4c46-a508-775157cda6ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:28.205300 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ecf365-3451-4c46-a508-775157cda6ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:28.207833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ecf365-3451-4c46-a508-775157cda6ad service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:28.277351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 912/1821] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 513 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:28.278481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 909/1821] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:28.295272 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a0356-faa6-428d-8c99-221f0ba9a9f6 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:28.382907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:28.438614 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a0356-faa6-428d-8c99-221f0ba9a9f6 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:28.451177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975dfa0f-1b25-4ae8-84e6-1dc1a078505b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:28.451896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975dfa0f-1b25-4ae8-84e6-1dc1a078505b admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:28.452174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975dfa0f-1b25-4ae8-84e6-1dc1a078505b admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:28.452445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975dfa0f-1b25-4ae8-84e6-1dc1a078505b admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:28.455311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975dfa0f-1b25-4ae8-84e6-1dc1a078505b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:28.459618 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975dfa0f-1b25-4ae8-84e6-1dc1a078505b admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:28.517631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 913/1822] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:28 2022] POST /identity/v3/projects => generated 385 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:28.521679 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecf365-3451-4c46-a508-775157cda6ad service nova] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:28.539665 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ecf365-3451-4c46-a508-775157cda6ad service nova] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:28.555052 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:28.598148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975dfa0f-1b25-4ae8-84e6-1dc1a078505b admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:28.600402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-975dfa0f-1b25-4ae8-84e6-1dc1a078505b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:28.712789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d4c5d-cfd1-491f-8bc8-3b28415eb1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:28.713616 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d4c5d-cfd1-491f-8bc8-3b28415eb1fe admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:28.713945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d4c5d-cfd1-491f-8bc8-3b28415eb1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:28.714507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d4c5d-cfd1-491f-8bc8-3b28415eb1fe admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:28.715355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d4c5d-cfd1-491f-8bc8-3b28415eb1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:28.717490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156d4c5d-cfd1-491f-8bc8-3b28415eb1fe admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:28.756120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 910/1823] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:28 2022] POST /identity/v3/users => generated 299 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:28.759219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d4c5d-cfd1-491f-8bc8-3b28415eb1fe admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:28.786545 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d4c5d-cfd1-491f-8bc8-3b28415eb1fe admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:28.799926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72fd5d-b9fc-4e5f-a884-a2544a960b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:28.808803 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72fd5d-b9fc-4e5f-a884-a2544a960b03 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:28.810403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72fd5d-b9fc-4e5f-a884-a2544a960b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:28.811399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72fd5d-b9fc-4e5f-a884-a2544a960b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:28.813418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72fd5d-b9fc-4e5f-a884-a2544a960b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:28.816124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72fd5d-b9fc-4e5f-a884-a2544a960b03 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:28.839157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975dfa0f-1b25-4ae8-84e6-1dc1a078505b admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:28.849896 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 911/1824] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:08:28.851222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72fd5d-b9fc-4e5f-a884-a2544a960b03 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:28.853729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:28.855056 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:28.855531 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:28.856380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9 admin admin] PATH_INFO: `/v3/projects/4753b7d382494a92a44700f226519147` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:28.857789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4753b7d382494a92a44700f226519147)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:28.859907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:28.863991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72fd5d-b9fc-4e5f-a884-a2544a960b03 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:28.882889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24a94a-ea3b-4862-b1d0-8ab268d90c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:28.883653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24a94a-ea3b-4862-b1d0-8ab268d90c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:28.883955 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24a94a-ea3b-4862-b1d0-8ab268d90c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:28.884368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24a94a-ea3b-4862-b1d0-8ab268d90c97 admin admin] PATH_INFO: `/v3/projects/2d11a233f8ad42b4a14dd2b5aa07d831/users/3050ae625c9c4c2c876fdb502a9f8530/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:28.922360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24a94a-ea3b-4862-b1d0-8ab268d90c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d11a233f8ad42b4a14dd2b5aa07d831, user_id=3050ae625c9c4c2c876fdb502a9f8530, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:28.925946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b24a94a-ea3b-4862-b1d0-8ab268d90c97 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:28.970095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4753b7d382494a92a44700f226519147', 'request_id': 'req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:28.983568 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4753b7d382494a92a44700f226519147', 'request_id': 'req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:28.989669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 912/1825] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:28 2022] PUT /identity/v3/projects/2d11a233f8ad42b4a14dd2b5aa07d831/users/3050ae625c9c4c2c876fdb502a9f8530/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:28.991415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24a94a-ea3b-4862-b1d0-8ab268d90c97 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:28.993814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4753b7d382494a92a44700f226519147', 'request_id': 'req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:29.020716 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9 admin admin] The token cache is being invalidate because project 4753b7d382494a92a44700f226519147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:29.020913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:29.021581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:29.023464 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 914/1826] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:28 2022] DELETE /identity/v3/projects/4753b7d382494a92a44700f226519147 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:29.036100 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:29.039589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:29.151922 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:29.169856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9aa11-0d5e-4a2b-8c2b-3def0cdb74b9 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:29.286380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7992442-ad0e-47cf-990c-4af0c50a25cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:29.287796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7992442-ad0e-47cf-990c-4af0c50a25cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:29.288289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7992442-ad0e-47cf-990c-4af0c50a25cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:29.288780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7992442-ad0e-47cf-990c-4af0c50a25cd admin admin] PATH_INFO: `/v3/users/713f610eec854e60a8fb3edb0adc227f` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:29.291780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7992442-ad0e-47cf-990c-4af0c50a25cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=713f610eec854e60a8fb3edb0adc227f)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:29.294780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7992442-ad0e-47cf-990c-4af0c50a25cd admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:29.300194 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24a94a-ea3b-4862-b1d0-8ab268d90c97 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:29.306344 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b4e29-53b7-4d25-aa0e-a256d3f3c0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:29.307064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b4e29-53b7-4d25-aa0e-a256d3f3c0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:29.307375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b4e29-53b7-4d25-aa0e-a256d3f3c0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:29.307749 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b4e29-53b7-4d25-aa0e-a256d3f3c0d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:29.308461 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b4e29-53b7-4d25-aa0e-a256d3f3c0d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:29.311163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b4e29-53b7-4d25-aa0e-a256d3f3c0d0 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:29.333018 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 913/1827] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:29.339933 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b4e29-53b7-4d25-aa0e-a256d3f3c0d0 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:29.344231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d01b4e29-53b7-4d25-aa0e-a256d3f3c0d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:29.377263 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b4e29-53b7-4d25-aa0e-a256d3f3c0d0 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:29.394897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edaf3a-3a60-42a3-a9b4-ee5dcd358872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:29.397722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edaf3a-3a60-42a3-a9b4-ee5dcd358872 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:29.399265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edaf3a-3a60-42a3-a9b4-ee5dcd358872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:29.401982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edaf3a-3a60-42a3-a9b4-ee5dcd358872 admin admin] PATH_INFO: `/v3/projects/2d11a233f8ad42b4a14dd2b5aa07d831/users/3050ae625c9c4c2c876fdb502a9f8530/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:29.408869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edaf3a-3a60-42a3-a9b4-ee5dcd358872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d11a233f8ad42b4a14dd2b5aa07d831, user_id=3050ae625c9c4c2c876fdb502a9f8530, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:29.417793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edaf3a-3a60-42a3-a9b4-ee5dcd358872 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:29.424886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c7992442-ad0e-47cf-990c-4af0c50a25cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '713f610eec854e60a8fb3edb0adc227f'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:29.442405 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c7992442-ad0e-47cf-990c-4af0c50a25cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '713f610eec854e60a8fb3edb0adc227f'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:29.444342 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c7992442-ad0e-47cf-990c-4af0c50a25cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '713f610eec854e60a8fb3edb0adc227f'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:29.446936 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c7992442-ad0e-47cf-990c-4af0c50a25cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '713f610eec854e60a8fb3edb0adc227f'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:29.450830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 914/1828] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:29 2022] PUT /identity/v3/projects/2d11a233f8ad42b4a14dd2b5aa07d831/users/3050ae625c9c4c2c876fdb502a9f8530/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:29.464765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ac299-b651-4f3f-bf57-82bbf1fd339c None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:29.467976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ac299-b651-4f3f-bf57-82bbf1fd339c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:29.468774 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ac299-b651-4f3f-bf57-82bbf1fd339c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:29.496458 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c7992442-ad0e-47cf-990c-4af0c50a25cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '713f610eec854e60a8fb3edb0adc227f'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:29.535358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 915/1829] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:29 2022] DELETE /identity/v3/users/713f610eec854e60a8fb3edb0adc227f => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:29.547823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-a89ac299-b651-4f3f-bf57-82bbf1fd339c None None] MFA Rules not processed for user `3050ae625c9c4c2c876fdb502a9f8530`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:29.550009 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a89ac299-b651-4f3f-bf57-82bbf1fd339c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:29.631382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:29.752175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 915/1830] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:29 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:29.820913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7992442-ad0e-47cf-990c-4af0c50a25cd admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:29.824884 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c7992442-ad0e-47cf-990c-4af0c50a25cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:30.001525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7992442-ad0e-47cf-990c-4af0c50a25cd admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:30.008124 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e420d-bf3d-4a07-97ba-53cb74466b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:30.009442 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e420d-bf3d-4a07-97ba-53cb74466b30 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:30.009992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e420d-bf3d-4a07-97ba-53cb74466b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:30.011395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e420d-bf3d-4a07-97ba-53cb74466b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:30.034265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e420d-bf3d-4a07-97ba-53cb74466b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:30.036812 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e420d-bf3d-4a07-97ba-53cb74466b30 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:30.058436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 916/1831] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:29 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:30.183511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ac299-b651-4f3f-bf57-82bbf1fd339c None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:30.185171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a89ac299-b651-4f3f-bf57-82bbf1fd339c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:30.338179 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ac299-b651-4f3f-bf57-82bbf1fd339c None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:30.344920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad939a5-b8da-4387-8bde-bfd8ef299c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:30.345886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad939a5-b8da-4387-8bde-bfd8ef299c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:30.346974 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad939a5-b8da-4387-8bde-bfd8ef299c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:30.347527 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad939a5-b8da-4387-8bde-bfd8ef299c05 admin admin] PATH_INFO: `/v3/users/53329b1a7ac54bdfa452d876d5c44fc1` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:30.349191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad939a5-b8da-4387-8bde-bfd8ef299c05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53329b1a7ac54bdfa452d876d5c44fc1)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:30.351923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad939a5-b8da-4387-8bde-bfd8ef299c05 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:30.389114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e420d-bf3d-4a07-97ba-53cb74466b30 service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:30.390741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a49e420d-bf3d-4a07-97ba-53cb74466b30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:30.435773 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e420d-bf3d-4a07-97ba-53cb74466b30 service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:30.458607 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4afc0-2235-4a14-b4c8-a61933e114ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:30.459621 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4afc0-2235-4a14-b4c8-a61933e114ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:30.461534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4afc0-2235-4a14-b4c8-a61933e114ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:30.461534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4afc0-2235-4a14-b4c8-a61933e114ae admin admin] PATH_INFO: `/v3/projects/b069c83932a94a9c84c3a7a58465fc7c` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:30.467797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4afc0-2235-4a14-b4c8-a61933e114ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=b069c83932a94a9c84c3a7a58465fc7c)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:30.474387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4afc0-2235-4a14-b4c8-a61933e114ae admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:30.478215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2ad939a5-b8da-4387-8bde-bfd8ef299c05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53329b1a7ac54bdfa452d876d5c44fc1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:30.500601 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2ad939a5-b8da-4387-8bde-bfd8ef299c05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53329b1a7ac54bdfa452d876d5c44fc1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:30.501723 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2ad939a5-b8da-4387-8bde-bfd8ef299c05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53329b1a7ac54bdfa452d876d5c44fc1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:30.503020 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2ad939a5-b8da-4387-8bde-bfd8ef299c05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53329b1a7ac54bdfa452d876d5c44fc1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:30.524184 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2ad939a5-b8da-4387-8bde-bfd8ef299c05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53329b1a7ac54bdfa452d876d5c44fc1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:30.530473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:30.566758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 916/1832] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:30 2022] DELETE /identity/v3/users/53329b1a7ac54bdfa452d876d5c44fc1 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:30.574232 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d1cd4-29b2-4daa-8134-7051a6030262 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:30.574514 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d1cd4-29b2-4daa-8134-7051a6030262 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:30.575486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d1cd4-29b2-4daa-8134-7051a6030262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:30.607593 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-ea4d1cd4-29b2-4daa-8134-7051a6030262 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:30.609080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ea4d1cd4-29b2-4daa-8134-7051a6030262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:30.614013 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-2ec4afc0-2235-4a14-b4c8-a61933e114ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b069c83932a94a9c84c3a7a58465fc7c', 'request_id': 'req-2ec4afc0-2235-4a14-b4c8-a61933e114ae'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:30.632670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-2ec4afc0-2235-4a14-b4c8-a61933e114ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b069c83932a94a9c84c3a7a58465fc7c', 'request_id': 'req-2ec4afc0-2235-4a14-b4c8-a61933e114ae'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:30.641017 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-2ec4afc0-2235-4a14-b4c8-a61933e114ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b069c83932a94a9c84c3a7a58465fc7c', 'request_id': 'req-2ec4afc0-2235-4a14-b4c8-a61933e114ae'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:30.668659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-2ec4afc0-2235-4a14-b4c8-a61933e114ae admin admin] The token cache is being invalidate because project b069c83932a94a9c84c3a7a58465fc7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:30.669008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-2ec4afc0-2235-4a14-b4c8-a61933e114ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:30.669760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-2ec4afc0-2235-4a14-b4c8-a61933e114ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:30.671863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 917/1833] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:30 2022] DELETE /identity/v3/projects/b069c83932a94a9c84c3a7a58465fc7c => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:30.675920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4afc0-2235-4a14-b4c8-a61933e114ae admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:30.678397 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2ec4afc0-2235-4a14-b4c8-a61933e114ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:30.683300 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:30.707926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 917/1834] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:30.743165 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d1cd4-29b2-4daa-8134-7051a6030262 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:30.750851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ea4d1cd4-29b2-4daa-8134-7051a6030262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:30.772772 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4afc0-2235-4a14-b4c8-a61933e114ae admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:30.783512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570b75b-b139-4c23-986f-9365c6718209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:30.784543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570b75b-b139-4c23-986f-9365c6718209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:30.784830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570b75b-b139-4c23-986f-9365c6718209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:30.785263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570b75b-b139-4c23-986f-9365c6718209 admin admin] PATH_INFO: `/v3/users/8f97a8ea0a2740ba8f095759a42530ed` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:30.787406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570b75b-b139-4c23-986f-9365c6718209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f97a8ea0a2740ba8f095759a42530ed)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:30.789707 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570b75b-b139-4c23-986f-9365c6718209 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:30.909445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-6570b75b-b139-4c23-986f-9365c6718209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f97a8ea0a2740ba8f095759a42530ed'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:30.916653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d1cd4-29b2-4daa-8134-7051a6030262 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:30.928666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b581e-5c24-409a-b9e7-5238d01621a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:30.929762 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b581e-5c24-409a-b9e7-5238d01621a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:30.930256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b581e-5c24-409a-b9e7-5238d01621a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:30.930729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989b581e-5c24-409a-b9e7-5238d01621a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:30.938048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-6570b75b-b139-4c23-986f-9365c6718209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f97a8ea0a2740ba8f095759a42530ed'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:30.939175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-6570b75b-b139-4c23-986f-9365c6718209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f97a8ea0a2740ba8f095759a42530ed'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:30.939834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-6570b75b-b139-4c23-986f-9365c6718209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f97a8ea0a2740ba8f095759a42530ed'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:30.949680 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b581e-5c24-409a-b9e7-5238d01621a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:30.952145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989b581e-5c24-409a-b9e7-5238d01621a3 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:30.962457 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-6570b75b-b139-4c23-986f-9365c6718209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f97a8ea0a2740ba8f095759a42530ed'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:30.981305 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 918/1835] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:30.982403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b581e-5c24-409a-b9e7-5238d01621a3 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:30.985443 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-989b581e-5c24-409a-b9e7-5238d01621a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:31.000680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 918/1836] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:30 2022] DELETE /identity/v3/users/8f97a8ea0a2740ba8f095759a42530ed => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:31.016636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989b581e-5c24-409a-b9e7-5238d01621a3 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:31.125394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a8c0b-a133-46a4-ad1f-2dfacc76a42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:31.126377 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a8c0b-a133-46a4-ad1f-2dfacc76a42f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:31.126818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a8c0b-a133-46a4-ad1f-2dfacc76a42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:31.127239 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a8c0b-a133-46a4-ad1f-2dfacc76a42f admin admin] PATH_INFO: `/v3/projects/585945b6362b4f47b19019499cedbcdf` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:31.128845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a8c0b-a133-46a4-ad1f-2dfacc76a42f admin admin] RBAC: Authorizing `identity:delete_project(project_id=585945b6362b4f47b19019499cedbcdf)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:31.131119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a8c0b-a133-46a4-ad1f-2dfacc76a42f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:31.202974 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cb0a8c0b-a133-46a4-ad1f-2dfacc76a42f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '585945b6362b4f47b19019499cedbcdf', 'request_id': 'req-cb0a8c0b-a133-46a4-ad1f-2dfacc76a42f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:31.219123 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cb0a8c0b-a133-46a4-ad1f-2dfacc76a42f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '585945b6362b4f47b19019499cedbcdf', 'request_id': 'req-cb0a8c0b-a133-46a4-ad1f-2dfacc76a42f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:31.226266 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cb0a8c0b-a133-46a4-ad1f-2dfacc76a42f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '585945b6362b4f47b19019499cedbcdf', 'request_id': 'req-cb0a8c0b-a133-46a4-ad1f-2dfacc76a42f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:31.238833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cb0a8c0b-a133-46a4-ad1f-2dfacc76a42f admin admin] The token cache is being invalidate because project 585945b6362b4f47b19019499cedbcdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:31.239134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cb0a8c0b-a133-46a4-ad1f-2dfacc76a42f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:31.239804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cb0a8c0b-a133-46a4-ad1f-2dfacc76a42f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:31.241693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 919/1837] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:30 2022] DELETE /identity/v3/projects/585945b6362b4f47b19019499cedbcdf => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:31.250481 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9b419-6250-4538-81b1-93ee1cd6d5c3 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:31.250875 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9b419-6250-4538-81b1-93ee1cd6d5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:31.251276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9b419-6250-4538-81b1-93ee1cd6d5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:31.276803 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-4ad9b419-6250-4538-81b1-93ee1cd6d5c3 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:31.278514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-4ad9b419-6250-4538-81b1-93ee1cd6d5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:31.297952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:31.360978 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 919/1838] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:31.369567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d8643-661b-4f88-94e7-86dad7a2ba8a None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:31.369907 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d8643-661b-4f88-94e7-86dad7a2ba8a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:31.370233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d8643-661b-4f88-94e7-86dad7a2ba8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:31.394620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-056d8643-661b-4f88-94e7-86dad7a2ba8a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:31.396160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-056d8643-661b-4f88-94e7-86dad7a2ba8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:31.411624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9b419-6250-4538-81b1-93ee1cd6d5c3 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:31.418472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 920/1839] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:31.422632 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9b419-6250-4538-81b1-93ee1cd6d5c3 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:31.428535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4b333-3c95-4adf-88d5-8417d5c9cede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:31.429201 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4b333-3c95-4adf-88d5-8417d5c9cede admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:31.429572 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4b333-3c95-4adf-88d5-8417d5c9cede admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:31.429933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4b333-3c95-4adf-88d5-8417d5c9cede admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:31.430806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4b333-3c95-4adf-88d5-8417d5c9cede admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:31.431197 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4b333-3c95-4adf-88d5-8417d5c9cede admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:31.433172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4b333-3c95-4adf-88d5-8417d5c9cede admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:31.451476 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 920/1840] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:08:31.459917 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d8643-661b-4f88-94e7-86dad7a2ba8a None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:31.463351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d8643-661b-4f88-94e7-86dad7a2ba8a None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:31.467622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633291af-3ba3-471b-b06c-26519dfa22fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:31.468243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633291af-3ba3-471b-b06c-26519dfa22fd admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:31.468520 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633291af-3ba3-471b-b06c-26519dfa22fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:31.468785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633291af-3ba3-471b-b06c-26519dfa22fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:31.469463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633291af-3ba3-471b-b06c-26519dfa22fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:31.471353 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633291af-3ba3-471b-b06c-26519dfa22fd admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:31.489042 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:31.512114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4b333-3c95-4adf-88d5-8417d5c9cede admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:31.515541 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 921/1841] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:31 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:31.523124 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4b333-3c95-4adf-88d5-8417d5c9cede admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:31.598430 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a5378-fcff-44b9-a5de-69cb88cfb8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:31.599579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a5378-fcff-44b9-a5de-69cb88cfb8b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:31.599878 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a5378-fcff-44b9-a5de-69cb88cfb8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:31.600141 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a5378-fcff-44b9-a5de-69cb88cfb8b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:31.600787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a5378-fcff-44b9-a5de-69cb88cfb8b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:31.602971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a5378-fcff-44b9-a5de-69cb88cfb8b3 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:31.631513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 921/1842] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:31 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:31.634835 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633291af-3ba3-471b-b06c-26519dfa22fd admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:31.649215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633291af-3ba3-471b-b06c-26519dfa22fd admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:31.668441 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6405f59-0e80-4d5c-a5f1-0b8a1b525c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:31.669088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6405f59-0e80-4d5c-a5f1-0b8a1b525c7a admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:31.669364 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6405f59-0e80-4d5c-a5f1-0b8a1b525c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:31.669642 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6405f59-0e80-4d5c-a5f1-0b8a1b525c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:31.670318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6405f59-0e80-4d5c-a5f1-0b8a1b525c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:31.673322 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6405f59-0e80-4d5c-a5f1-0b8a1b525c7a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:31.686789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 922/1843] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:31.691362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a5378-fcff-44b9-a5de-69cb88cfb8b3 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:31.702875 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a5378-fcff-44b9-a5de-69cb88cfb8b3 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:31.709293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6405f59-0e80-4d5c-a5f1-0b8a1b525c7a admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:31.711554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b6405f59-0e80-4d5c-a5f1-0b8a1b525c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:31.719571 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de26fc-55fd-4d87-a905-f2f13c5cb6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:31.720219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de26fc-55fd-4d87-a905-f2f13c5cb6e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:31.720733 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de26fc-55fd-4d87-a905-f2f13c5cb6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:31.727490 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de26fc-55fd-4d87-a905-f2f13c5cb6e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:31.728588 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de26fc-55fd-4d87-a905-f2f13c5cb6e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:31.730893 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de26fc-55fd-4d87-a905-f2f13c5cb6e7 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:31.735001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6405f59-0e80-4d5c-a5f1-0b8a1b525c7a admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:31.741269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb54e6-6ea0-41ce-a79b-d68104142c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:31.742975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb54e6-6ea0-41ce-a79b-d68104142c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:31.743302 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb54e6-6ea0-41ce-a79b-d68104142c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:31.743516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb54e6-6ea0-41ce-a79b-d68104142c2b admin admin] PATH_INFO: `/v3/projects/d26c35b041b54f609b9e2e4dff4cb0f7` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:31.744898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb54e6-6ea0-41ce-a79b-d68104142c2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d26c35b041b54f609b9e2e4dff4cb0f7)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:31.747994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb54e6-6ea0-41ce-a79b-d68104142c2b admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:31.760965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 922/1844] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:31.769186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de26fc-55fd-4d87-a905-f2f13c5cb6e7 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:31.795529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de26fc-55fd-4d87-a905-f2f13c5cb6e7 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:31.800795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130d74f-9fdc-43c7-9aec-6338abc01739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:31.801475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130d74f-9fdc-43c7-9aec-6338abc01739 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:31.802215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130d74f-9fdc-43c7-9aec-6338abc01739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:31.802215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130d74f-9fdc-43c7-9aec-6338abc01739 admin admin] PATH_INFO: `/v3/projects/7f37bacffce84c44b2403e3909166913/users/75bd13701d364ac38f687dd3a74e300d/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:31.827289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b5cb54e6-6ea0-41ce-a79b-d68104142c2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd26c35b041b54f609b9e2e4dff4cb0f7', 'request_id': 'req-b5cb54e6-6ea0-41ce-a79b-d68104142c2b'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:31.832788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130d74f-9fdc-43c7-9aec-6338abc01739 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f37bacffce84c44b2403e3909166913, user_id=75bd13701d364ac38f687dd3a74e300d, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:31.835638 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130d74f-9fdc-43c7-9aec-6338abc01739 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:31.836517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b5cb54e6-6ea0-41ce-a79b-d68104142c2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd26c35b041b54f609b9e2e4dff4cb0f7', 'request_id': 'req-b5cb54e6-6ea0-41ce-a79b-d68104142c2b'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:31.844694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b5cb54e6-6ea0-41ce-a79b-d68104142c2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd26c35b041b54f609b9e2e4dff4cb0f7', 'request_id': 'req-b5cb54e6-6ea0-41ce-a79b-d68104142c2b'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:31.852158 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 923/1845] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:31 2022] PUT /identity/v3/projects/7f37bacffce84c44b2403e3909166913/users/75bd13701d364ac38f687dd3a74e300d/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:31.859185 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b5cb54e6-6ea0-41ce-a79b-d68104142c2b admin admin] The token cache is being invalidate because project d26c35b041b54f609b9e2e4dff4cb0f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:31.859507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b5cb54e6-6ea0-41ce-a79b-d68104142c2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:31.860188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b5cb54e6-6ea0-41ce-a79b-d68104142c2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:31.861409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130d74f-9fdc-43c7-9aec-6338abc01739 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:31.862159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 923/1846] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:31 2022] DELETE /identity/v3/projects/d26c35b041b54f609b9e2e4dff4cb0f7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:31.863432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99594100-a5f6-4951-9148-24a0022c0bc2 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:31.863774 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99594100-a5f6-4951-9148-24a0022c0bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:31.864019 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99594100-a5f6-4951-9148-24a0022c0bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:31.870609 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8130d74f-9fdc-43c7-9aec-6338abc01739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:31.900176 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-99594100-a5f6-4951-9148-24a0022c0bc2 None None] MFA Rules not processed for user `75bd13701d364ac38f687dd3a74e300d`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:31.901644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-99594100-a5f6-4951-9148-24a0022c0bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:31.965306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130d74f-9fdc-43c7-9aec-6338abc01739 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:31.972608 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a601f7-be8d-40a9-8614-727596e3e3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:31.973428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a601f7-be8d-40a9-8614-727596e3e3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:31.973854 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a601f7-be8d-40a9-8614-727596e3e3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:31.974332 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a601f7-be8d-40a9-8614-727596e3e3fc admin admin] PATH_INFO: `/v3/projects/ded4735dd38e4a7882360256d1b2c7fc` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:31.975977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a601f7-be8d-40a9-8614-727596e3e3fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ded4735dd38e4a7882360256d1b2c7fc)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:31.979278 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a601f7-be8d-40a9-8614-727596e3e3fc admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:31.987382 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:31.999973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 924/1847] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:32.001017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99594100-a5f6-4951-9148-24a0022c0bc2 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:32.003090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-99594100-a5f6-4951-9148-24a0022c0bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:32.018294 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99594100-a5f6-4951-9148-24a0022c0bc2 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:32.023915 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a5d4b-2f4a-48cf-9be8-0faebbcd5a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:32.024831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a5d4b-2f4a-48cf-9be8-0faebbcd5a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.025295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a5d4b-2f4a-48cf-9be8-0faebbcd5a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.025727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a5d4b-2f4a-48cf-9be8-0faebbcd5a61 admin admin] PATH_INFO: `/v3/users/3050ae625c9c4c2c876fdb502a9f8530` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.027180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a5d4b-2f4a-48cf-9be8-0faebbcd5a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3050ae625c9c4c2c876fdb502a9f8530)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:32.029597 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a5d4b-2f4a-48cf-9be8-0faebbcd5a61 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:32.058959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-08a601f7-be8d-40a9-8614-727596e3e3fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ded4735dd38e4a7882360256d1b2c7fc', 'request_id': 'req-08a601f7-be8d-40a9-8614-727596e3e3fc'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.069910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-08a601f7-be8d-40a9-8614-727596e3e3fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ded4735dd38e4a7882360256d1b2c7fc', 'request_id': 'req-08a601f7-be8d-40a9-8614-727596e3e3fc'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.077945 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-08a601f7-be8d-40a9-8614-727596e3e3fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ded4735dd38e4a7882360256d1b2c7fc', 'request_id': 'req-08a601f7-be8d-40a9-8614-727596e3e3fc'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.089679 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-300a5d4b-2f4a-48cf-9be8-0faebbcd5a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3050ae625c9c4c2c876fdb502a9f8530'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.092485 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:INSERT=3,keystone:SELECT=83,keystone:DELETE=14 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:32.095767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-08a601f7-be8d-40a9-8614-727596e3e3fc admin admin] The token cache is being invalidate because project ded4735dd38e4a7882360256d1b2c7fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:32.096302 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-08a601f7-be8d-40a9-8614-727596e3e3fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.102592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-08a601f7-be8d-40a9-8614-727596e3e3fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.103797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 924/1848] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:31 2022] DELETE /identity/v3/projects/ded4735dd38e4a7882360256d1b2c7fc => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:32.105619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a601f7-be8d-40a9-8614-727596e3e3fc admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:32.108088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-08a601f7-be8d-40a9-8614-727596e3e3fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:32.113160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-300a5d4b-2f4a-48cf-9be8-0faebbcd5a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3050ae625c9c4c2c876fdb502a9f8530'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.113988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-300a5d4b-2f4a-48cf-9be8-0faebbcd5a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3050ae625c9c4c2c876fdb502a9f8530'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.114845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-300a5d4b-2f4a-48cf-9be8-0faebbcd5a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3050ae625c9c4c2c876fdb502a9f8530'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.124180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-300a5d4b-2f4a-48cf-9be8-0faebbcd5a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3050ae625c9c4c2c876fdb502a9f8530'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.146295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 925/1849] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:32 2022] DELETE /identity/v3/users/3050ae625c9c4c2c876fdb502a9f8530 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:32.147614 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48ca4d-e0d8-4ced-8afd-6dab72d5283a None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.147902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48ca4d-e0d8-4ced-8afd-6dab72d5283a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.148170 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48ca4d-e0d8-4ced-8afd-6dab72d5283a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.169326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-1a48ca4d-e0d8-4ced-8afd-6dab72d5283a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:32.170532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1a48ca4d-e0d8-4ced-8afd-6dab72d5283a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:32.178783 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a601f7-be8d-40a9-8614-727596e3e3fc admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:32.241930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89555e58-03bc-4fcc-9d01-199c55e2058d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:32.242613 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89555e58-03bc-4fcc-9d01-199c55e2058d service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.242948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89555e58-03bc-4fcc-9d01-199c55e2058d service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.243539 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89555e58-03bc-4fcc-9d01-199c55e2058d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.245878 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-89555e58-03bc-4fcc-9d01-199c55e2058d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:32.249694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 926/1850] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:32.258937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1bc8b-3ee0-4b1b-9f93-0468d28cdfd9 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.259155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1bc8b-3ee0-4b1b-9f93-0468d28cdfd9 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.259430 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1bc8b-3ee0-4b1b-9f93-0468d28cdfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.286501 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-12b1bc8b-3ee0-4b1b-9f93-0468d28cdfd9 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:32.287819 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-12b1bc8b-3ee0-4b1b-9f93-0468d28cdfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:32.301839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 927/1851] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:32.306173 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1bc8b-3ee0-4b1b-9f93-0468d28cdfd9 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:32.315705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1bc8b-3ee0-4b1b-9f93-0468d28cdfd9 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:32.322373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89555e58-03bc-4fcc-9d01-199c55e2058d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:32.323974 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89555e58-03bc-4fcc-9d01-199c55e2058d service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:32.332136 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9c5e2-e847-4617-b334-ad19ef91e163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:32.332725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9c5e2-e847-4617-b334-ad19ef91e163 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.333007 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9c5e2-e847-4617-b334-ad19ef91e163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.333280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9c5e2-e847-4617-b334-ad19ef91e163 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.333938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 925/1852] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:32.334165 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9c5e2-e847-4617-b334-ad19ef91e163 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:32.334253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9c5e2-e847-4617-b334-ad19ef91e163 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:32.335891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9c5e2-e847-4617-b334-ad19ef91e163 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:32.349775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 928/1853] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:08:32.354649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89555e58-03bc-4fcc-9d01-199c55e2058d service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:32.357872 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9c5e2-e847-4617-b334-ad19ef91e163 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:32.360157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c4c9c5e2-e847-4617-b334-ad19ef91e163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:32.363653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:32.370383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89555e58-03bc-4fcc-9d01-199c55e2058d service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:32.376837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b672b-271a-4a37-86cf-284d61bf17e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:32.377532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b672b-271a-4a37-86cf-284d61bf17e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.377885 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b672b-271a-4a37-86cf-284d61bf17e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.378065 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b672b-271a-4a37-86cf-284d61bf17e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.379119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9c5e2-e847-4617-b334-ad19ef91e163 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:32.379856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b672b-271a-4a37-86cf-284d61bf17e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:32.383223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b672b-271a-4a37-86cf-284d61bf17e7 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:32.385935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cce2aa-4cb1-44cf-b46e-d28daf01c19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:32.386898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cce2aa-4cb1-44cf-b46e-d28daf01c19c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.387544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cce2aa-4cb1-44cf-b46e-d28daf01c19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.387923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cce2aa-4cb1-44cf-b46e-d28daf01c19c admin admin] PATH_INFO: `/v3/users/75bd13701d364ac38f687dd3a74e300d` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.389575 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cce2aa-4cb1-44cf-b46e-d28daf01c19c admin admin] RBAC: Authorizing `identity:delete_user(user_id=75bd13701d364ac38f687dd3a74e300d)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:32.392010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cce2aa-4cb1-44cf-b46e-d28daf01c19c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:32.422451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 926/1854] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:32 2022] POST /identity/v3/projects => generated 399 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:32.427239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b672b-271a-4a37-86cf-284d61bf17e7 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:32.452248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e9cce2aa-4cb1-44cf-b46e-d28daf01c19c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75bd13701d364ac38f687dd3a74e300d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.476479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e9cce2aa-4cb1-44cf-b46e-d28daf01c19c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75bd13701d364ac38f687dd3a74e300d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.477661 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e9cce2aa-4cb1-44cf-b46e-d28daf01c19c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75bd13701d364ac38f687dd3a74e300d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.478554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e9cce2aa-4cb1-44cf-b46e-d28daf01c19c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75bd13701d364ac38f687dd3a74e300d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.493524 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e9cce2aa-4cb1-44cf-b46e-d28daf01c19c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75bd13701d364ac38f687dd3a74e300d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.508832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b672b-271a-4a37-86cf-284d61bf17e7 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:32.514218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9b726-aa88-4241-a7b0-88e2876b5ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:32.514907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9b726-aa88-4241-a7b0-88e2876b5ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.515245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9b726-aa88-4241-a7b0-88e2876b5ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.515653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 929/1855] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:32 2022] DELETE /identity/v3/users/75bd13701d364ac38f687dd3a74e300d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:32.515765 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9b726-aa88-4241-a7b0-88e2876b5ec6 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.516410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9b726-aa88-4241-a7b0-88e2876b5ec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:32.516523 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cce2aa-4cb1-44cf-b46e-d28daf01c19c admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:32.518019 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e9cce2aa-4cb1-44cf-b46e-d28daf01c19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:32.518659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9b726-aa88-4241-a7b0-88e2876b5ec6 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:32.549631 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:32.564984 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 927/1856] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:32 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:08:32.566428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a532a-e4af-4f87-8c8d-5bab38ecc534 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.566711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a532a-e4af-4f87-8c8d-5bab38ecc534 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.566983 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a532a-e4af-4f87-8c8d-5bab38ecc534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.601405 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-994a532a-e4af-4f87-8c8d-5bab38ecc534 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:32.603313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-994a532a-e4af-4f87-8c8d-5bab38ecc534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:32.603515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cce2aa-4cb1-44cf-b46e-d28daf01c19c admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:32.608645 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fdff9-9330-4c37-a014-238697371ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:32.609411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fdff9-9330-4c37-a014-238697371ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.609752 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fdff9-9330-4c37-a014-238697371ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.610083 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fdff9-9330-4c37-a014-238697371ef1 admin admin] PATH_INFO: `/v3/projects/2d11a233f8ad42b4a14dd2b5aa07d831` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.611335 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fdff9-9330-4c37-a014-238697371ef1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d11a233f8ad42b4a14dd2b5aa07d831)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:32.613416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fdff9-9330-4c37-a014-238697371ef1 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:32.625571 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 928/1857] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:32.626508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a532a-e4af-4f87-8c8d-5bab38ecc534 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:32.627911 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-994a532a-e4af-4f87-8c8d-5bab38ecc534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:32.641792 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a532a-e4af-4f87-8c8d-5bab38ecc534 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:32.648206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553a921-a99a-4930-a4f8-10717da6ccb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:32.648930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553a921-a99a-4930-a4f8-10717da6ccb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.649219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553a921-a99a-4930-a4f8-10717da6ccb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.649502 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c553a921-a99a-4930-a4f8-10717da6ccb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.650194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c553a921-a99a-4930-a4f8-10717da6ccb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:32.652050 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c553a921-a99a-4930-a4f8-10717da6ccb7 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:32.663435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 929/1858] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:32.664657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553a921-a99a-4930-a4f8-10717da6ccb7 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:32.666847 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c553a921-a99a-4930-a4f8-10717da6ccb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:32.670330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a63fdff9-9330-4c37-a014-238697371ef1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d11a233f8ad42b4a14dd2b5aa07d831', 'request_id': 'req-a63fdff9-9330-4c37-a014-238697371ef1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.681649 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a63fdff9-9330-4c37-a014-238697371ef1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d11a233f8ad42b4a14dd2b5aa07d831', 'request_id': 'req-a63fdff9-9330-4c37-a014-238697371ef1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.689641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a63fdff9-9330-4c37-a014-238697371ef1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d11a233f8ad42b4a14dd2b5aa07d831', 'request_id': 'req-a63fdff9-9330-4c37-a014-238697371ef1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.703415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a63fdff9-9330-4c37-a014-238697371ef1 admin admin] The token cache is being invalidate because project 2d11a233f8ad42b4a14dd2b5aa07d831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:32.703956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a63fdff9-9330-4c37-a014-238697371ef1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.704684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a63fdff9-9330-4c37-a014-238697371ef1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:32.706387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 930/1859] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:32 2022] DELETE /identity/v3/projects/2d11a233f8ad42b4a14dd2b5aa07d831 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:32.707312 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fdff9-9330-4c37-a014-238697371ef1 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:32.710222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a63fdff9-9330-4c37-a014-238697371ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:32.735338 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c553a921-a99a-4930-a4f8-10717da6ccb7 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:32.779314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fdff9-9330-4c37-a014-238697371ef1 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:32.785070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5517b-ebd2-4a22-ba1c-2756f8b30402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:32.785807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5517b-ebd2-4a22-ba1c-2756f8b30402 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.786180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5517b-ebd2-4a22-ba1c-2756f8b30402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.786546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5517b-ebd2-4a22-ba1c-2756f8b30402 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.787332 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5517b-ebd2-4a22-ba1c-2756f8b30402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:32.789076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5517b-ebd2-4a22-ba1c-2756f8b30402 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:32.791142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c588a00-c5eb-459d-b5ab-ea9460d9d562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:32.792151 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c588a00-c5eb-459d-b5ab-ea9460d9d562 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.792436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c588a00-c5eb-459d-b5ab-ea9460d9d562 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.792706 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c588a00-c5eb-459d-b5ab-ea9460d9d562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.795037 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3c588a00-c5eb-459d-b5ab-ea9460d9d562 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:32.804538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 931/1860] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:32.807096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8043c-367d-465a-ae58-0b6e913d0994 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.807644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8043c-367d-465a-ae58-0b6e913d0994 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.807923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab8043c-367d-465a-ae58-0b6e913d0994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.810367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c588a00-c5eb-459d-b5ab-ea9460d9d562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:32.812573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c588a00-c5eb-459d-b5ab-ea9460d9d562 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:32.824352 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 930/1861] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:32.825212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c588a00-c5eb-459d-b5ab-ea9460d9d562 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:32.829358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-aab8043c-367d-465a-ae58-0b6e913d0994 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:32.830556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-aab8043c-367d-465a-ae58-0b6e913d0994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:32.835925 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c588a00-c5eb-459d-b5ab-ea9460d9d562 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:32.838743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 932/1862] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:32.841453 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5c5b3-66eb-4e82-b216-f8813bfb7607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:32.842193 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5c5b3-66eb-4e82-b216-f8813bfb7607 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.842705 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5c5b3-66eb-4e82-b216-f8813bfb7607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.842927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5c5b3-66eb-4e82-b216-f8813bfb7607 admin admin] PATH_INFO: `/v3/projects/b89bb001554444c5b200d9b6c0ea95e0/users/bd2298664242483cb083c85df077d351/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.846163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffcab3-5cf4-4c38-802f-bb796d574e51 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.846403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffcab3-5cf4-4c38-802f-bb796d574e51 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.846672 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffcab3-5cf4-4c38-802f-bb796d574e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.859796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:32.864908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd5c5b3-66eb-4e82-b216-f8813bfb7607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b89bb001554444c5b200d9b6c0ea95e0, user_id=bd2298664242483cb083c85df077d351, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:32.867980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-09ffcab3-5cf4-4c38-802f-bb796d574e51 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:32.868846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd5c5b3-66eb-4e82-b216-f8813bfb7607 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:32.869285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-09ffcab3-5cf4-4c38-802f-bb796d574e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:32.878146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 933/1863] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:32.883085 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffcab3-5cf4-4c38-802f-bb796d574e51 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:32.896685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 931/1864] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:32 2022] PUT /identity/v3/projects/b89bb001554444c5b200d9b6c0ea95e0/users/bd2298664242483cb083c85df077d351/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:32.901079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee91c3a-d1c0-42c6-aff4-947f37eea66e None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.901577 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee91c3a-d1c0-42c6-aff4-947f37eea66e None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.901982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee91c3a-d1c0-42c6-aff4-947f37eea66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.902570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffcab3-5cf4-4c38-802f-bb796d574e51 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:32.932197 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-2ee91c3a-d1c0-42c6-aff4-947f37eea66e None None] MFA Rules not processed for user `bd2298664242483cb083c85df077d351`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:32.933473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2ee91c3a-d1c0-42c6-aff4-947f37eea66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:32.941890 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:32.977694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf6f21-288a-4c24-9e6b-161763e538fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:32.978323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf6f21-288a-4c24-9e6b-161763e538fc admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:32.978591 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf6f21-288a-4c24-9e6b-161763e538fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:32.978856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf6f21-288a-4c24-9e6b-161763e538fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:32.979600 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf6f21-288a-4c24-9e6b-161763e538fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:32.979900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf6f21-288a-4c24-9e6b-161763e538fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:32.981546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf6f21-288a-4c24-9e6b-161763e538fc admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:32.996358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 934/1865] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:08:33.002351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf6f21-288a-4c24-9e6b-161763e538fc admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.015670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf6f21-288a-4c24-9e6b-161763e538fc admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.021367 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a340e33-91f7-4a98-bba2-55cc5d00949d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.021983 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a340e33-91f7-4a98-bba2-55cc5d00949d admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.022684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a340e33-91f7-4a98-bba2-55cc5d00949d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.022684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a340e33-91f7-4a98-bba2-55cc5d00949d admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.023424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a340e33-91f7-4a98-bba2-55cc5d00949d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.025730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a340e33-91f7-4a98-bba2-55cc5d00949d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.031301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 932/1866] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:33.035429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee91c3a-d1c0-42c6-aff4-947f37eea66e None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.053203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee91c3a-d1c0-42c6-aff4-947f37eea66e None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.058836 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a89741-e6b6-4ada-b85b-5491eb99e9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.059562 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a89741-e6b6-4ada-b85b-5491eb99e9a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.059847 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a89741-e6b6-4ada-b85b-5491eb99e9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.060122 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a89741-e6b6-4ada-b85b-5491eb99e9a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.060755 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a89741-e6b6-4ada-b85b-5491eb99e9a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.062732 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a89741-e6b6-4ada-b85b-5491eb99e9a9 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.071633 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 935/1867] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:33 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:33.072839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a340e33-91f7-4a98-bba2-55cc5d00949d admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.099091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 933/1868] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:33 2022] POST /identity/v3/projects => generated 401 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:33.103039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a89741-e6b6-4ada-b85b-5491eb99e9a9 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.180758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a340e33-91f7-4a98-bba2-55cc5d00949d admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.207284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a89741-e6b6-4ada-b85b-5491eb99e9a9 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.212003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b079a-f229-40ad-9098-f164d3e41db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.212623 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b079a-f229-40ad-9098-f164d3e41db9 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.212896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b079a-f229-40ad-9098-f164d3e41db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.213161 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b079a-f229-40ad-9098-f164d3e41db9 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.213852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b079a-f229-40ad-9098-f164d3e41db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.215859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b079a-f229-40ad-9098-f164d3e41db9 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.239487 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 934/1869] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:33 2022] POST /identity/v3/users => generated 307 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:08:33.240008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b079a-f229-40ad-9098-f164d3e41db9 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.244564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3d7b079a-f229-40ad-9098-f164d3e41db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:33.249915 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceeea3-f655-452b-ad29-8d6992451862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.250663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceeea3-f655-452b-ad29-8d6992451862 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.251030 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceeea3-f655-452b-ad29-8d6992451862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.251586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dceeea3-f655-452b-ad29-8d6992451862 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.252166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dceeea3-f655-452b-ad29-8d6992451862 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.254332 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dceeea3-f655-452b-ad29-8d6992451862 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.257648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b079a-f229-40ad-9098-f164d3e41db9 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.263776 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b8924-192c-428b-9a19-62173c3f9dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.264056 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b8924-192c-428b-9a19-62173c3f9dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.264330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b8924-192c-428b-9a19-62173c3f9dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.264596 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b8924-192c-428b-9a19-62173c3f9dbe admin admin] PATH_INFO: `/v3/projects/7f37bacffce84c44b2403e3909166913` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.265828 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b8924-192c-428b-9a19-62173c3f9dbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f37bacffce84c44b2403e3909166913)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.267971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b8924-192c-428b-9a19-62173c3f9dbe admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.281370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 936/1870] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:33 2022] POST /identity/v3/users => generated 290 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:33.282417 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceeea3-f655-452b-ad29-8d6992451862 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.293189 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dceeea3-f655-452b-ad29-8d6992451862 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.300882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a45dd-e31d-4c11-bf2d-320652828b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.301624 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a45dd-e31d-4c11-bf2d-320652828b77 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.302014 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a45dd-e31d-4c11-bf2d-320652828b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.302218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a45dd-e31d-4c11-bf2d-320652828b77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.302881 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a45dd-e31d-4c11-bf2d-320652828b77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.304612 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a45dd-e31d-4c11-bf2d-320652828b77 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.314854 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:33.323229 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c23b8924-192c-428b-9a19-62173c3f9dbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f37bacffce84c44b2403e3909166913', 'request_id': 'req-c23b8924-192c-428b-9a19-62173c3f9dbe'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:33.326892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 937/1871] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:33.327878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a45dd-e31d-4c11-bf2d-320652828b77 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.332097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c23b8924-192c-428b-9a19-62173c3f9dbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f37bacffce84c44b2403e3909166913', 'request_id': 'req-c23b8924-192c-428b-9a19-62173c3f9dbe'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:33.338765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a45dd-e31d-4c11-bf2d-320652828b77 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.339116 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c23b8924-192c-428b-9a19-62173c3f9dbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f37bacffce84c44b2403e3909166913', 'request_id': 'req-c23b8924-192c-428b-9a19-62173c3f9dbe'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:33.352369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c23b8924-192c-428b-9a19-62173c3f9dbe admin admin] The token cache is being invalidate because project 7f37bacffce84c44b2403e3909166913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:33.352571 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c23b8924-192c-428b-9a19-62173c3f9dbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:33.353263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c23b8924-192c-428b-9a19-62173c3f9dbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:33.354931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 935/1872] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:33 2022] DELETE /identity/v3/projects/7f37bacffce84c44b2403e3909166913 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:33.355747 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b8924-192c-428b-9a19-62173c3f9dbe admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.357039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c23b8924-192c-428b-9a19-62173c3f9dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:33.372877 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:33.402041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b9b01-0bd9-4c05-8903-80158e325487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.402631 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b9b01-0bd9-4c05-8903-80158e325487 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.402908 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b9b01-0bd9-4c05-8903-80158e325487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.403180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b9b01-0bd9-4c05-8903-80158e325487 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.403855 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b9b01-0bd9-4c05-8903-80158e325487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.405447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b9b01-0bd9-4c05-8903-80158e325487 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.415080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 938/1873] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:33.416417 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5b1a1-1872-4adc-9ae9-7cc12ca4b207 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.416684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5b1a1-1872-4adc-9ae9-7cc12ca4b207 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.416986 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5b1a1-1872-4adc-9ae9-7cc12ca4b207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.420703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b8924-192c-428b-9a19-62173c3f9dbe admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.425253 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3fdd15-450a-425b-9a98-3aff96d7d3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.425928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fdd15-450a-425b-9a98-3aff96d7d3b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.426210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fdd15-450a-425b-9a98-3aff96d7d3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.426481 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3fdd15-450a-425b-9a98-3aff96d7d3b2 admin admin] PATH_INFO: `/v3/projects/6f1b167073aa447f8f2dee124b733c81/users/324bc0f422ed4f59a03ec9f0f0828f21/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.436323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-e2e5b1a1-1872-4adc-9ae9-7cc12ca4b207 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:33.437536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e2e5b1a1-1872-4adc-9ae9-7cc12ca4b207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:33.441462 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3fdd15-450a-425b-9a98-3aff96d7d3b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f1b167073aa447f8f2dee124b733c81, user_id=324bc0f422ed4f59a03ec9f0f0828f21, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.444220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3fdd15-450a-425b-9a98-3aff96d7d3b2 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.446247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 939/1874] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:33.446837 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5b1a1-1872-4adc-9ae9-7cc12ca4b207 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.448036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e2e5b1a1-1872-4adc-9ae9-7cc12ca4b207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:33.460019 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5b1a1-1872-4adc-9ae9-7cc12ca4b207 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.461185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 936/1875] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:33 2022] PUT /identity/v3/projects/6f1b167073aa447f8f2dee124b733c81/users/324bc0f422ed4f59a03ec9f0f0828f21/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:33.462467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50194ce6-79c3-48a0-9bd8-2cdbeb46c58c None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.462910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50194ce6-79c3-48a0-9bd8-2cdbeb46c58c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.463187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50194ce6-79c3-48a0-9bd8-2cdbeb46c58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.484374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-50194ce6-79c3-48a0-9bd8-2cdbeb46c58c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:33.485544 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-50194ce6-79c3-48a0-9bd8-2cdbeb46c58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:33.520697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98793da9-b345-4dc3-95e9-8b8881627504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.521374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98793da9-b345-4dc3-95e9-8b8881627504 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.521579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98793da9-b345-4dc3-95e9-8b8881627504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.521882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98793da9-b345-4dc3-95e9-8b8881627504 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.522522 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98793da9-b345-4dc3-95e9-8b8881627504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.524317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98793da9-b345-4dc3-95e9-8b8881627504 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.534497 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 940/1876] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:33.535571 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98793da9-b345-4dc3-95e9-8b8881627504 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.545355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98793da9-b345-4dc3-95e9-8b8881627504 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.548343 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 937/1877] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:33.549177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50194ce6-79c3-48a0-9bd8-2cdbeb46c58c None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.550165 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ecf66-a740-4a23-8f8c-76ad6308d5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.550754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ecf66-a740-4a23-8f8c-76ad6308d5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.551069 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ecf66-a740-4a23-8f8c-76ad6308d5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.551532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ecf66-a740-4a23-8f8c-76ad6308d5e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.552236 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ecf66-a740-4a23-8f8c-76ad6308d5e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.553840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ecf66-a740-4a23-8f8c-76ad6308d5e4 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.558544 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50194ce6-79c3-48a0-9bd8-2cdbeb46c58c None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.563735 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac67bb2-8cb3-4665-b0cf-0d3623517d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.564013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 941/1878] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:33.564756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac67bb2-8cb3-4665-b0cf-0d3623517d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.564907 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ecf66-a740-4a23-8f8c-76ad6308d5e4 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.565157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac67bb2-8cb3-4665-b0cf-0d3623517d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.565604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac67bb2-8cb3-4665-b0cf-0d3623517d28 admin admin] PATH_INFO: `/v3/projects/15a749b00d444437a70af3a0b2f87926/users/70aac88799e14a7d8cbdbba993e3e9a9/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.574692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ecf66-a740-4a23-8f8c-76ad6308d5e4 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.579351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3a1e8-df4e-4667-be98-6d465b79a432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.579960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3a1e8-df4e-4667-be98-6d465b79a432 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.580234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3a1e8-df4e-4667-be98-6d465b79a432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.580505 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3a1e8-df4e-4667-be98-6d465b79a432 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.581131 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3a1e8-df4e-4667-be98-6d465b79a432 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:33.581428 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3a1e8-df4e-4667-be98-6d465b79a432 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.581715 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac67bb2-8cb3-4665-b0cf-0d3623517d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15a749b00d444437a70af3a0b2f87926, user_id=70aac88799e14a7d8cbdbba993e3e9a9, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.582980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3a1e8-df4e-4667-be98-6d465b79a432 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.584619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac67bb2-8cb3-4665-b0cf-0d3623517d28 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.596343 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 942/1879] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:08:33.597163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3a1e8-df4e-4667-be98-6d465b79a432 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.598841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 938/1880] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:33 2022] PUT /identity/v3/projects/15a749b00d444437a70af3a0b2f87926/users/70aac88799e14a7d8cbdbba993e3e9a9/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:33.600429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac67bb2-8cb3-4665-b0cf-0d3623517d28 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.610312 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3a1e8-df4e-4667-be98-6d465b79a432 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.635576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:33.676336 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac67bb2-8cb3-4665-b0cf-0d3623517d28 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.681381 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0a930-1071-466c-a6f5-c20a3541ceb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.681875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b5f46-819d-4bf4-a83b-44e32f6011d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.681996 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0a930-1071-466c-a6f5-c20a3541ceb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.682266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0a930-1071-466c-a6f5-c20a3541ceb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.682577 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b5f46-819d-4bf4-a83b-44e32f6011d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.682690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0a930-1071-466c-a6f5-c20a3541ceb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.682858 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b5f46-819d-4bf4-a83b-44e32f6011d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.683128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b5f46-819d-4bf4-a83b-44e32f6011d3 admin admin] PATH_INFO: `/v3/projects/6f1b167073aa447f8f2dee124b733c81/users/324bc0f422ed4f59a03ec9f0f0828f21/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.683283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0a930-1071-466c-a6f5-c20a3541ceb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.685053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b5f46-819d-4bf4-a83b-44e32f6011d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f1b167073aa447f8f2dee124b733c81, user_id=324bc0f422ed4f59a03ec9f0f0828f21, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.685261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0a930-1071-466c-a6f5-c20a3541ceb8 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.687701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5b5f46-819d-4bf4-a83b-44e32f6011d3 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.705030 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 943/1881] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:33 2022] PUT /identity/v3/projects/6f1b167073aa447f8f2dee124b733c81/users/324bc0f422ed4f59a03ec9f0f0828f21/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:33.705030 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cc4e0-6780-4183-9b5b-e4940a8e2e46 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.705030 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cc4e0-6780-4183-9b5b-e4940a8e2e46 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.705030 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324cc4e0-6780-4183-9b5b-e4940a8e2e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.718290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 939/1882] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:33 2022] POST /identity/v3/projects => generated 361 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:08:33.724733 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078142a3-cfd7-48e6-bf40-73ba38a28bd6 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.725112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078142a3-cfd7-48e6-bf40-73ba38a28bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.725560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078142a3-cfd7-48e6-bf40-73ba38a28bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.739778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-324cc4e0-6780-4183-9b5b-e4940a8e2e46 None None] MFA Rules not processed for user `70aac88799e14a7d8cbdbba993e3e9a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:33.741035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-324cc4e0-6780-4183-9b5b-e4940a8e2e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:33.758077 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-078142a3-cfd7-48e6-bf40-73ba38a28bd6 None None] MFA Rules not processed for user `324bc0f422ed4f59a03ec9f0f0828f21`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:33.759851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-078142a3-cfd7-48e6-bf40-73ba38a28bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:33.767034 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:33.852365 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 940/1883] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:33 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:33.853547 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078142a3-cfd7-48e6-bf40-73ba38a28bd6 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.854749 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 944/1884] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:33.858885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cc4e0-6780-4183-9b5b-e4940a8e2e46 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.935817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324cc4e0-6780-4183-9b5b-e4940a8e2e46 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.941622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201e9fa-a446-4d51-9523-8e8fc0ee66a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.942266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078142a3-cfd7-48e6-bf40-73ba38a28bd6 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:33.942448 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201e9fa-a446-4d51-9523-8e8fc0ee66a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.942857 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201e9fa-a446-4d51-9523-8e8fc0ee66a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.943121 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201e9fa-a446-4d51-9523-8e8fc0ee66a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.943877 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201e9fa-a446-4d51-9523-8e8fc0ee66a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.946292 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201e9fa-a446-4d51-9523-8e8fc0ee66a7 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.947939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79d712-ea0b-4576-8d90-c7588f7f1705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:33.948791 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79d712-ea0b-4576-8d90-c7588f7f1705 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:33.949079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79d712-ea0b-4576-8d90-c7588f7f1705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:33.949444 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79d712-ea0b-4576-8d90-c7588f7f1705 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:33.950142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79d712-ea0b-4576-8d90-c7588f7f1705 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:33.952158 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79d712-ea0b-4576-8d90-c7588f7f1705 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:33.977641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 945/1885] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:33 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:33.978508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201e9fa-a446-4d51-9523-8e8fc0ee66a7 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.982806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 941/1886] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:33 2022] POST /identity/v3/users => generated 281 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:33.982952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b201e9fa-a446-4d51-9523-8e8fc0ee66a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:33.983033 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79d712-ea0b-4576-8d90-c7588f7f1705 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:33.991972 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79d712-ea0b-4576-8d90-c7588f7f1705 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:34.023759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:34.040398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201e9fa-a446-4d51-9523-8e8fc0ee66a7 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:34.045346 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd97d9-ea7e-4d88-a495-5944b43c6157 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:34.045346 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd97d9-ea7e-4d88-a495-5944b43c6157 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.045606 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd97d9-ea7e-4d88-a495-5944b43c6157 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.045910 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd97d9-ea7e-4d88-a495-5944b43c6157 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.060232 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c027e53-bd30-4988-b415-9e4e67fc8227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:34.060836 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c027e53-bd30-4988-b415-9e4e67fc8227 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.061494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c027e53-bd30-4988-b415-9e4e67fc8227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.061494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c027e53-bd30-4988-b415-9e4e67fc8227 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.062052 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c027e53-bd30-4988-b415-9e4e67fc8227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:34.064104 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c027e53-bd30-4988-b415-9e4e67fc8227 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:34.088146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 942/1887] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:33 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:08:34.089107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c027e53-bd30-4988-b415-9e4e67fc8227 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:34.098434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c027e53-bd30-4988-b415-9e4e67fc8227 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:34.102748 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:34.103329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac344a-7acf-40f1-b10f-ce9311b2f513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:34.104021 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac344a-7acf-40f1-b10f-ce9311b2f513 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.104336 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac344a-7acf-40f1-b10f-ce9311b2f513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.104708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac344a-7acf-40f1-b10f-ce9311b2f513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.105640 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac344a-7acf-40f1-b10f-ce9311b2f513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:34.107209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac344a-7acf-40f1-b10f-ce9311b2f513 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:34.113869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd97d9-ea7e-4d88-a495-5944b43c6157 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:34.115319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd97d9-ea7e-4d88-a495-5944b43c6157 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:34.118205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 943/1888] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:34.119145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac344a-7acf-40f1-b10f-ce9311b2f513 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:34.125585 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 946/1889] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:33 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:34.126594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd97d9-ea7e-4d88-a495-5944b43c6157 service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:34.129196 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac344a-7acf-40f1-b10f-ce9311b2f513 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:34.134797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f912c-f750-4009-b3e9-90426a21cbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:34.135596 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f912c-f750-4009-b3e9-90426a21cbfd admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.136180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f912c-f750-4009-b3e9-90426a21cbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.136613 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f912c-f750-4009-b3e9-90426a21cbfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.136725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd97d9-ea7e-4d88-a495-5944b43c6157 service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:34.137637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f912c-f750-4009-b3e9-90426a21cbfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:34.139511 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f912c-f750-4009-b3e9-90426a21cbfd admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:34.141500 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28cbb1-082c-4f82-8ba7-5bcb6d577bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:34.142891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28cbb1-082c-4f82-8ba7-5bcb6d577bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.143239 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28cbb1-082c-4f82-8ba7-5bcb6d577bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.143536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28cbb1-082c-4f82-8ba7-5bcb6d577bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.144321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f28cbb1-082c-4f82-8ba7-5bcb6d577bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:34.146155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f28cbb1-082c-4f82-8ba7-5bcb6d577bb1 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:34.152297 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 944/1890] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:34.156388 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f912c-f750-4009-b3e9-90426a21cbfd admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:34.157340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 947/1891] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:34.164194 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28cbb1-082c-4f82-8ba7-5bcb6d577bb1 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:34.170271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f912c-f750-4009-b3e9-90426a21cbfd admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:34.175666 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ea429-58fa-4f8f-bc36-02b8c127ccd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:34.176404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ea429-58fa-4f8f-bc36-02b8c127ccd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.176763 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ea429-58fa-4f8f-bc36-02b8c127ccd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.177093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ea429-58fa-4f8f-bc36-02b8c127ccd4 admin admin] PATH_INFO: `/v3/projects/a9c2fd03d8054685920873b34856731c/users/87620e57a6ff46148fb550b7922bf2b4/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.182684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28cbb1-082c-4f82-8ba7-5bcb6d577bb1 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:34.187925 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02dd341-7fd0-426b-81ac-a59fa8d4f165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:34.188643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dd341-7fd0-426b-81ac-a59fa8d4f165 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.189029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dd341-7fd0-426b-81ac-a59fa8d4f165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.189280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dd341-7fd0-426b-81ac-a59fa8d4f165 admin admin] PATH_INFO: `/v3/projects/82f4bbc218a44a929db485f7fc8b6d51/users/5c3942d549454087acfe5226135dc5c8/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.206232 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ea429-58fa-4f8f-bc36-02b8c127ccd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9c2fd03d8054685920873b34856731c, user_id=87620e57a6ff46148fb550b7922bf2b4, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:34.206232 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ea429-58fa-4f8f-bc36-02b8c127ccd4 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:34.207170 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02dd341-7fd0-426b-81ac-a59fa8d4f165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f4bbc218a44a929db485f7fc8b6d51, user_id=5c3942d549454087acfe5226135dc5c8, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:34.210241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02dd341-7fd0-426b-81ac-a59fa8d4f165 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:34.226451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 945/1892] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:34 2022] PUT /identity/v3/projects/a9c2fd03d8054685920873b34856731c/users/87620e57a6ff46148fb550b7922bf2b4/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:34.226562 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 948/1893] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:34 2022] PUT /identity/v3/projects/82f4bbc218a44a929db485f7fc8b6d51/users/5c3942d549454087acfe5226135dc5c8/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:34.227599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ea429-58fa-4f8f-bc36-02b8c127ccd4 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:34.233394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0091b-c110-4b9f-970d-072ee7ddad2b None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.234405 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0091b-c110-4b9f-970d-072ee7ddad2b None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.234405 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec0091b-c110-4b9f-970d-072ee7ddad2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.262448 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-3ec0091b-c110-4b9f-970d-072ee7ddad2b None None] MFA Rules not processed for user `5c3942d549454087acfe5226135dc5c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:34.263868 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3ec0091b-c110-4b9f-970d-072ee7ddad2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:34.289578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ea429-58fa-4f8f-bc36-02b8c127ccd4 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:34.333110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 949/1894] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:34.333994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0091b-c110-4b9f-970d-072ee7ddad2b None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:34.363894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbb135-778d-4aea-865e-11ababb36267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:34.364876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbb135-778d-4aea-865e-11ababb36267 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.365323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbb135-778d-4aea-865e-11ababb36267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.365779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dbb135-778d-4aea-865e-11ababb36267 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.366717 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbb135-778d-4aea-865e-11ababb36267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:34.368532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dbb135-778d-4aea-865e-11ababb36267 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:34.385011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 946/1895] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:34.386301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbb135-778d-4aea-865e-11ababb36267 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:34.388037 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-85dbb135-778d-4aea-865e-11ababb36267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:34.406490 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:34.406863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec0091b-c110-4b9f-970d-072ee7ddad2b None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:34.411046 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf60bc1-6a21-4603-919b-b019eb8f5e7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:34.411789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf60bc1-6a21-4603-919b-b019eb8f5e7f service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.412075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf60bc1-6a21-4603-919b-b019eb8f5e7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.412430 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf60bc1-6a21-4603-919b-b019eb8f5e7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.414224 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-eaf60bc1-6a21-4603-919b-b019eb8f5e7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:34.453741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dbb135-778d-4aea-865e-11ababb36267 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:34.458915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3b9a2-7c67-4ace-a456-b1367eb3c26d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:34.459669 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3b9a2-7c67-4ace-a456-b1367eb3c26d service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.460025 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3b9a2-7c67-4ace-a456-b1367eb3c26d service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.460361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c3b9a2-7c67-4ace-a456-b1367eb3c26d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.469396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3b9a2-7c67-4ace-a456-b1367eb3c26d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:34.470755 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c3b9a2-7c67-4ace-a456-b1367eb3c26d service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:34.477868 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf60bc1-6a21-4603-919b-b019eb8f5e7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:34.479311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf60bc1-6a21-4603-919b-b019eb8f5e7f service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:34.480862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 947/1896] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:34.484244 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:34.485113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3b9a2-7c67-4ace-a456-b1367eb3c26d service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:34.499732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 950/1897] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:34 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:34.508789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf60bc1-6a21-4603-919b-b019eb8f5e7f service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:34.514543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c3b9a2-7c67-4ace-a456-b1367eb3c26d service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:34.519635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66817d2b-cc06-4a48-a50d-f1e199ee1e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:34.520405 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66817d2b-cc06-4a48-a50d-f1e199ee1e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.520757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66817d2b-cc06-4a48-a50d-f1e199ee1e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.521080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf60bc1-6a21-4603-919b-b019eb8f5e7f service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:34.521180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66817d2b-cc06-4a48-a50d-f1e199ee1e20 admin admin] PATH_INFO: `/v3/projects/a9c2fd03d8054685920873b34856731c/users/87620e57a6ff46148fb550b7922bf2b4/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.523120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66817d2b-cc06-4a48-a50d-f1e199ee1e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9c2fd03d8054685920873b34856731c, user_id=87620e57a6ff46148fb550b7922bf2b4, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:34.525834 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66bb007-228d-4dc7-87db-22963d5d6ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:34.526217 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66817d2b-cc06-4a48-a50d-f1e199ee1e20 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:34.526538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bb007-228d-4dc7-87db-22963d5d6ee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.526813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bb007-228d-4dc7-87db-22963d5d6ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.527083 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66bb007-228d-4dc7-87db-22963d5d6ee2 admin admin] PATH_INFO: `/v3/users/5c3942d549454087acfe5226135dc5c8` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.528267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bb007-228d-4dc7-87db-22963d5d6ee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c3942d549454087acfe5226135dc5c8)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:34.530327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66bb007-228d-4dc7-87db-22963d5d6ee2 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:34.543500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 948/1898] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:34 2022] PUT /identity/v3/projects/a9c2fd03d8054685920873b34856731c/users/87620e57a6ff46148fb550b7922bf2b4/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:34.552491 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d5ae2-dbb2-4f1e-9650-40d8b0926d73 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.552776 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d5ae2-dbb2-4f1e-9650-40d8b0926d73 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.553051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d5ae2-dbb2-4f1e-9650-40d8b0926d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.586195 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f66bb007-228d-4dc7-87db-22963d5d6ee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c3942d549454087acfe5226135dc5c8'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:34.586430 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-b24d5ae2-dbb2-4f1e-9650-40d8b0926d73 None None] MFA Rules not processed for user `87620e57a6ff46148fb550b7922bf2b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:34.587525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b24d5ae2-dbb2-4f1e-9650-40d8b0926d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:34.595705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f66bb007-228d-4dc7-87db-22963d5d6ee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c3942d549454087acfe5226135dc5c8'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:34.596448 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f66bb007-228d-4dc7-87db-22963d5d6ee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c3942d549454087acfe5226135dc5c8'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:34.597035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f66bb007-228d-4dc7-87db-22963d5d6ee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c3942d549454087acfe5226135dc5c8'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:34.605275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f66bb007-228d-4dc7-87db-22963d5d6ee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c3942d549454087acfe5226135dc5c8'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:34.630674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 951/1899] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:34 2022] DELETE /identity/v3/users/5c3942d549454087acfe5226135dc5c8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:34.632893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f220bc-efc4-4dcd-981d-b1d786503a78 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.633148 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f220bc-efc4-4dcd-981d-b1d786503a78 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.633428 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f220bc-efc4-4dcd-981d-b1d786503a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.657402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-83f220bc-efc4-4dcd-981d-b1d786503a78 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:34.658546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-83f220bc-efc4-4dcd-981d-b1d786503a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:34.668399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 949/1900] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:34 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:34.680750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d5ae2-dbb2-4f1e-9650-40d8b0926d73 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:34.682175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b24d5ae2-dbb2-4f1e-9650-40d8b0926d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:34.731220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 952/1901] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:34.740919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f220bc-efc4-4dcd-981d-b1d786503a78 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:34.742387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-83f220bc-efc4-4dcd-981d-b1d786503a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:34.756006 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d5ae2-dbb2-4f1e-9650-40d8b0926d73 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:34.812725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:34.819494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9f7e6-b9f4-4509-b5ed-4e1401628c9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:34.821708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9f7e6-b9f4-4509-b5ed-4e1401628c9e service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.822256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9f7e6-b9f4-4509-b5ed-4e1401628c9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.822552 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9f7e6-b9f4-4509-b5ed-4e1401628c9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.825118 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f220bc-efc4-4dcd-981d-b1d786503a78 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:34.829900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5bfbe-9396-440f-b666-3ce355f6a588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:34.830632 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5bfbe-9396-440f-b666-3ce355f6a588 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:34.830808 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5bfbe-9396-440f-b666-3ce355f6a588 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:34.831048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5bfbe-9396-440f-b666-3ce355f6a588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:34.841829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5bfbe-9396-440f-b666-3ce355f6a588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:34.844633 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d5bfbe-9396-440f-b666-3ce355f6a588 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:34.854165 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 953/1902] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:08:34.906103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9f7e6-b9f4-4509-b5ed-4e1401628c9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:34.907939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9f7e6-b9f4-4509-b5ed-4e1401628c9e service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:34.919160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 950/1903] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:34 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:08:35.391108 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5bfbe-9396-440f-b666-3ce355f6a588 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:35.403555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5bfbe-9396-440f-b666-3ce355f6a588 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:35.409959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f2717-95b5-43c2-912d-9cedef2d231b tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87620e57a6ff46148fb550b7922bf2b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9c2fd03d8054685920873b34856731c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:35.410567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f2717-95b5-43c2-912d-9cedef2d231b tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:35.410843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f2717-95b5-43c2-912d-9cedef2d231b tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:35.411124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f2717-95b5-43c2-912d-9cedef2d231b tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:35.411803 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f2717-95b5-43c2-912d-9cedef2d231b tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:35.412101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f2717-95b5-43c2-912d-9cedef2d231b tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:35.413697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f2717-95b5-43c2-912d-9cedef2d231b tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:35.428925 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 954/1904] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:08:35.432217 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9f7e6-b9f4-4509-b5ed-4e1401628c9e service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:35.444390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9f7e6-b9f4-4509-b5ed-4e1401628c9e service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:35.450157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eece790-05a8-4f25-ba3e-c4dd5457f50d tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87620e57a6ff46148fb550b7922bf2b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9c2fd03d8054685920873b34856731c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:35.450862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eece790-05a8-4f25-ba3e-c4dd5457f50d tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:35.451204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eece790-05a8-4f25-ba3e-c4dd5457f50d tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:35.451583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eece790-05a8-4f25-ba3e-c4dd5457f50d tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:35.452326 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eece790-05a8-4f25-ba3e-c4dd5457f50d tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:35.454306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eece790-05a8-4f25-ba3e-c4dd5457f50d tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:35.484671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 951/1905] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:35 2022] POST /identity/v3/projects => generated 355 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:35.486010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f2717-95b5-43c2-912d-9cedef2d231b tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:35.492224 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:35.498156 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f2717-95b5-43c2-912d-9cedef2d231b tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:35.505098 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eece790-05a8-4f25-ba3e-c4dd5457f50d tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:35.507654 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-7eece790-05a8-4f25-ba3e-c4dd5457f50d tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:35.583942 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55794236-9405-497a-af63-3cdb4fe2ef81 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87620e57a6ff46148fb550b7922bf2b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9c2fd03d8054685920873b34856731c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:35.584668 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55794236-9405-497a-af63-3cdb4fe2ef81 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:35.585020 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55794236-9405-497a-af63-3cdb4fe2ef81 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:35.585968 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55794236-9405-497a-af63-3cdb4fe2ef81 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:35.586707 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55794236-9405-497a-af63-3cdb4fe2ef81 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:35.587771 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55794236-9405-497a-af63-3cdb4fe2ef81 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:35.589486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55794236-9405-497a-af63-3cdb4fe2ef81 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:35.620082 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eece790-05a8-4f25-ba3e-c4dd5457f50d tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:35.623900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 955/1906] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:08:35.625195 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0d84f2-417e-4550-b353-1eddb39d1603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:35.625981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d84f2-417e-4550-b353-1eddb39d1603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:35.626933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d84f2-417e-4550-b353-1eddb39d1603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:35.626933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d84f2-417e-4550-b353-1eddb39d1603 admin admin] PATH_INFO: `/v3/projects/82f4bbc218a44a929db485f7fc8b6d51` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:35.628281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d84f2-417e-4550-b353-1eddb39d1603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82f4bbc218a44a929db485f7fc8b6d51)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:35.630331 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0d84f2-417e-4550-b353-1eddb39d1603 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:35.676736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55794236-9405-497a-af63-3cdb4fe2ef81 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:35.691131 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55794236-9405-497a-af63-3cdb4fe2ef81 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:35.707726 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-7d0d84f2-417e-4550-b353-1eddb39d1603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82f4bbc218a44a929db485f7fc8b6d51', 'request_id': 'req-7d0d84f2-417e-4550-b353-1eddb39d1603'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:35.723579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-7d0d84f2-417e-4550-b353-1eddb39d1603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82f4bbc218a44a929db485f7fc8b6d51', 'request_id': 'req-7d0d84f2-417e-4550-b353-1eddb39d1603'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:35.737841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-7d0d84f2-417e-4550-b353-1eddb39d1603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82f4bbc218a44a929db485f7fc8b6d51', 'request_id': 'req-7d0d84f2-417e-4550-b353-1eddb39d1603'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:35.751902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-7d0d84f2-417e-4550-b353-1eddb39d1603 admin admin] The token cache is being invalidate because project 82f4bbc218a44a929db485f7fc8b6d51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:35.752399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-7d0d84f2-417e-4550-b353-1eddb39d1603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:35.753475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-7d0d84f2-417e-4550-b353-1eddb39d1603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:35.759600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 952/1907] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:35 2022] DELETE /identity/v3/projects/82f4bbc218a44a929db485f7fc8b6d51 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:35.761557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5e1a4-b08c-47f6-9a0d-4aa5812cff7d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:35.761986 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5e1a4-b08c-47f6-9a0d-4aa5812cff7d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:35.762245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5e1a4-b08c-47f6-9a0d-4aa5812cff7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:35.773797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:35.777497 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68461582-6c51-4243-94c8-e447ddb80200 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87620e57a6ff46148fb550b7922bf2b4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a9c2fd03d8054685920873b34856731c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:35.778249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68461582-6c51-4243-94c8-e447ddb80200 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:35.778602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68461582-6c51-4243-94c8-e447ddb80200 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:35.778945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68461582-6c51-4243-94c8-e447ddb80200 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] PATH_INFO: `/v3/projects/1af48ff7ea544fc9931decc8abd0e4b1` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:35.780607 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68461582-6c51-4243-94c8-e447ddb80200 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1af48ff7ea544fc9931decc8abd0e4b1)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:35.782905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68461582-6c51-4243-94c8-e447ddb80200 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:35.787728 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-84d5e1a4-b08c-47f6-9a0d-4aa5812cff7d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:35.790419 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-84d5e1a4-b08c-47f6-9a0d-4aa5812cff7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:35.845047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-68461582-6c51-4243-94c8-e447ddb80200 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1af48ff7ea544fc9931decc8abd0e4b1', 'request_id': 'req-68461582-6c51-4243-94c8-e447ddb80200'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:35.857291 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-68461582-6c51-4243-94c8-e447ddb80200 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1af48ff7ea544fc9931decc8abd0e4b1', 'request_id': 'req-68461582-6c51-4243-94c8-e447ddb80200'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:35.865573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-68461582-6c51-4243-94c8-e447ddb80200 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1af48ff7ea544fc9931decc8abd0e4b1', 'request_id': 'req-68461582-6c51-4243-94c8-e447ddb80200'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:35.868832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 953/1908] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:35.876701 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158559b-b523-4c71-b1b9-3f9c4d3c6c06 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:35.877013 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158559b-b523-4c71-b1b9-3f9c4d3c6c06 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:35.877287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158559b-b523-4c71-b1b9-3f9c4d3c6c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:35.884530 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-68461582-6c51-4243-94c8-e447ddb80200 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] The token cache is being invalidate because project 1af48ff7ea544fc9931decc8abd0e4b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:35.885456 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-68461582-6c51-4243-94c8-e447ddb80200 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:35.885991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-68461582-6c51-4243-94c8-e447ddb80200 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:35.903528 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 956/1909] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:35 2022] DELETE /identity/v3/projects/1af48ff7ea544fc9931decc8abd0e4b1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:35.904180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-6158559b-b523-4c71-b1b9-3f9c4d3c6c06 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:35.904180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6158559b-b523-4c71-b1b9-3f9c4d3c6c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:35.992553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 954/1910] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:35.997647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68461582-6c51-4243-94c8-e447ddb80200 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:36.011171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68461582-6c51-4243-94c8-e447ddb80200 tempest-VolumeQuotaClassesTest-250055041 tempest-VolumeQuotaClassesTest-250055041-project-admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:36.017750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a111822-f600-4f68-a9c7-e2d8fba3c79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:36.018729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a111822-f600-4f68-a9c7-e2d8fba3c79c admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:36.019098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a111822-f600-4f68-a9c7-e2d8fba3c79c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:36.019438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a111822-f600-4f68-a9c7-e2d8fba3c79c admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:36.020232 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a111822-f600-4f68-a9c7-e2d8fba3c79c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:36.020601 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a111822-f600-4f68-a9c7-e2d8fba3c79c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:36.022462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a111822-f600-4f68-a9c7-e2d8fba3c79c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:36.037547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 957/1911] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:08:36.042543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158559b-b523-4c71-b1b9-3f9c4d3c6c06 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:36.052820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158559b-b523-4c71-b1b9-3f9c4d3c6c06 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:36.058767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab2c74-8056-4b57-8f59-ffa30f35b934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:36.059535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab2c74-8056-4b57-8f59-ffa30f35b934 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:36.060006 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab2c74-8056-4b57-8f59-ffa30f35b934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:36.060368 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab2c74-8056-4b57-8f59-ffa30f35b934 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:36.061332 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab2c74-8056-4b57-8f59-ffa30f35b934 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:36.064956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab2c74-8056-4b57-8f59-ffa30f35b934 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:36.117287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 955/1912] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:36 2022] POST /identity/v3/projects => generated 375 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:36.123234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a111822-f600-4f68-a9c7-e2d8fba3c79c admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:36.137402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a111822-f600-4f68-a9c7-e2d8fba3c79c admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:36.149788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:36.209281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab2c74-8056-4b57-8f59-ffa30f35b934 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:36.211574 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5cab2c74-8056-4b57-8f59-ffa30f35b934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:36.223317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0ddba-e803-4e0f-bb0e-378f171899ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:36.224048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0ddba-e803-4e0f-bb0e-378f171899ee admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:36.224370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0ddba-e803-4e0f-bb0e-378f171899ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:36.224587 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d0ddba-e803-4e0f-bb0e-378f171899ee admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:36.225241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0ddba-e803-4e0f-bb0e-378f171899ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:36.227171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d0ddba-e803-4e0f-bb0e-378f171899ee admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:36.256332 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 958/1913] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:36 2022] POST /identity/v3/users => generated 288 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:36.260302 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0ddba-e803-4e0f-bb0e-378f171899ee admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:36.273964 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d0ddba-e803-4e0f-bb0e-378f171899ee admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:36.279368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e445e-feb6-45fc-83de-82ab8853e624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:36.280157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e445e-feb6-45fc-83de-82ab8853e624 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:36.280437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e445e-feb6-45fc-83de-82ab8853e624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:36.280712 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e445e-feb6-45fc-83de-82ab8853e624 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:36.281749 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e445e-feb6-45fc-83de-82ab8853e624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:36.283647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e445e-feb6-45fc-83de-82ab8853e624 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:36.296390 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 959/1914] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:36.299547 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab2c74-8056-4b57-8f59-ffa30f35b934 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:36.304044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e445e-feb6-45fc-83de-82ab8853e624 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:36.305427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926be24-1905-4595-9295-f491aa3e9828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:36.306242 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926be24-1905-4595-9295-f491aa3e9828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:36.306661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926be24-1905-4595-9295-f491aa3e9828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:36.307096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926be24-1905-4595-9295-f491aa3e9828 admin admin] PATH_INFO: `/v3/users/70aac88799e14a7d8cbdbba993e3e9a9` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:36.315800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926be24-1905-4595-9295-f491aa3e9828 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70aac88799e14a7d8cbdbba993e3e9a9)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:36.315800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c926be24-1905-4595-9295-f491aa3e9828 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:36.316482 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e445e-feb6-45fc-83de-82ab8853e624 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:36.322103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ebddc-7e88-4ce5-ac98-0583192f238b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:36.324106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ebddc-7e88-4ce5-ac98-0583192f238b admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:36.324106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ebddc-7e88-4ce5-ac98-0583192f238b admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:36.324235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ebddc-7e88-4ce5-ac98-0583192f238b admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:36.324895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ebddc-7e88-4ce5-ac98-0583192f238b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:36.326481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ebddc-7e88-4ce5-ac98-0583192f238b admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:36.347680 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 960/1915] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:08:36.352486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ebddc-7e88-4ce5-ac98-0583192f238b admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:36.356946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:36.364761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ebddc-7e88-4ce5-ac98-0583192f238b admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:36.375471 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8789c-0fb5-45f3-b183-dd46dff90ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:36.376170 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8789c-0fb5-45f3-b183-dd46dff90ce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:36.376444 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8789c-0fb5-45f3-b183-dd46dff90ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:36.376721 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8789c-0fb5-45f3-b183-dd46dff90ce3 admin admin] PATH_INFO: `/v3/projects/5c011058726b4756b3178dbf5037739c/users/3db22bc465fa46a1a993e94599b1e3af/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:36.394573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8789c-0fb5-45f3-b183-dd46dff90ce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c011058726b4756b3178dbf5037739c, user_id=3db22bc465fa46a1a993e94599b1e3af, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:36.397312 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8789c-0fb5-45f3-b183-dd46dff90ce3 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:36.403308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c926be24-1905-4595-9295-f491aa3e9828 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70aac88799e14a7d8cbdbba993e3e9a9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:36.414389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c926be24-1905-4595-9295-f491aa3e9828 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70aac88799e14a7d8cbdbba993e3e9a9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:36.416316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c926be24-1905-4595-9295-f491aa3e9828 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70aac88799e14a7d8cbdbba993e3e9a9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:36.417157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 961/1916] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:36 2022] PUT /identity/v3/projects/5c011058726b4756b3178dbf5037739c/users/3db22bc465fa46a1a993e94599b1e3af/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:36.417305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c926be24-1905-4595-9295-f491aa3e9828 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70aac88799e14a7d8cbdbba993e3e9a9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:36.428934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98424f89-6f49-407f-a144-76cd661a1c69 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:36.429125 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98424f89-6f49-407f-a144-76cd661a1c69 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:36.429402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98424f89-6f49-407f-a144-76cd661a1c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:36.436522 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c926be24-1905-4595-9295-f491aa3e9828 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70aac88799e14a7d8cbdbba993e3e9a9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:36.465498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-98424f89-6f49-407f-a144-76cd661a1c69 None None] MFA Rules not processed for user `3db22bc465fa46a1a993e94599b1e3af`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:36.467501 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-98424f89-6f49-407f-a144-76cd661a1c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:36.473695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 956/1917] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:36 2022] DELETE /identity/v3/users/70aac88799e14a7d8cbdbba993e3e9a9 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:36.479700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21e782-f85e-41bf-8aa2-9d2ab5e75e57 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:36.480444 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21e782-f85e-41bf-8aa2-9d2ab5e75e57 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:36.480946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21e782-f85e-41bf-8aa2-9d2ab5e75e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:36.515268 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-ad21e782-f85e-41bf-8aa2-9d2ab5e75e57 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:36.517777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ad21e782-f85e-41bf-8aa2-9d2ab5e75e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:36.541569 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 962/1918] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:36.550808 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98424f89-6f49-407f-a144-76cd661a1c69 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:36.552418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-98424f89-6f49-407f-a144-76cd661a1c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:36.602661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 957/1919] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:36.613146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21e782-f85e-41bf-8aa2-9d2ab5e75e57 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:36.614835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ad21e782-f85e-41bf-8aa2-9d2ab5e75e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:36.622653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:36.627970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98424f89-6f49-407f-a144-76cd661a1c69 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:36.672765 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21e782-f85e-41bf-8aa2-9d2ab5e75e57 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:36.676346 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36446e6c-8b0e-47bb-8864-019aa1e1868a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:36.676959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36446e6c-8b0e-47bb-8864-019aa1e1868a service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:36.677228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36446e6c-8b0e-47bb-8864-019aa1e1868a service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:36.677494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36446e6c-8b0e-47bb-8864-019aa1e1868a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:36.680641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072f5de-180a-4904-b78e-f70cf7989aab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:36.681258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072f5de-180a-4904-b78e-f70cf7989aab service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:36.681528 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072f5de-180a-4904-b78e-f70cf7989aab service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:36.681789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072f5de-180a-4904-b78e-f70cf7989aab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:36.686452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36446e6c-8b0e-47bb-8864-019aa1e1868a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:36.687932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36446e6c-8b0e-47bb-8864-019aa1e1868a service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:36.697569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 958/1920] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:08:36.738382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c072f5de-180a-4904-b78e-f70cf7989aab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:36.740350 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c072f5de-180a-4904-b78e-f70cf7989aab service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:36.750865 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 963/1921] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:36.982528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36446e6c-8b0e-47bb-8864-019aa1e1868a service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:36.984294 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-36446e6c-8b0e-47bb-8864-019aa1e1868a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:36.999422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36446e6c-8b0e-47bb-8864-019aa1e1868a service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:37.004746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d92ef-d315-43f2-872a-e756a0e8b85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:37.005409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d92ef-d315-43f2-872a-e756a0e8b85f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:37.005690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d92ef-d315-43f2-872a-e756a0e8b85f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:37.006041 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37d92ef-d315-43f2-872a-e756a0e8b85f admin admin] PATH_INFO: `/v3/projects/15a749b00d444437a70af3a0b2f87926` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:37.007225 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d92ef-d315-43f2-872a-e756a0e8b85f admin admin] RBAC: Authorizing `identity:delete_project(project_id=15a749b00d444437a70af3a0b2f87926)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:37.009170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37d92ef-d315-43f2-872a-e756a0e8b85f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:37.057045 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:37.064661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f37d92ef-d315-43f2-872a-e756a0e8b85f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15a749b00d444437a70af3a0b2f87926', 'request_id': 'req-f37d92ef-d315-43f2-872a-e756a0e8b85f'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:37.075749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f37d92ef-d315-43f2-872a-e756a0e8b85f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15a749b00d444437a70af3a0b2f87926', 'request_id': 'req-f37d92ef-d315-43f2-872a-e756a0e8b85f'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:37.084358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f37d92ef-d315-43f2-872a-e756a0e8b85f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15a749b00d444437a70af3a0b2f87926', 'request_id': 'req-f37d92ef-d315-43f2-872a-e756a0e8b85f'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:37.099931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f37d92ef-d315-43f2-872a-e756a0e8b85f admin admin] The token cache is being invalidate because project 15a749b00d444437a70af3a0b2f87926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:37.100441 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f37d92ef-d315-43f2-872a-e756a0e8b85f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:37.101049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f37d92ef-d315-43f2-872a-e756a0e8b85f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:37.111973 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 959/1922] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:36 2022] DELETE /identity/v3/projects/15a749b00d444437a70af3a0b2f87926 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:37.112381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072f5de-180a-4904-b78e-f70cf7989aab service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:37.118107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072f5de-180a-4904-b78e-f70cf7989aab service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:37.176598 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac28661-3ae5-4f99-9af3-87466c878b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:37.177247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac28661-3ae5-4f99-9af3-87466c878b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:37.177521 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac28661-3ae5-4f99-9af3-87466c878b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:37.177789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac28661-3ae5-4f99-9af3-87466c878b59 admin admin] PATH_INFO: `/v3/users/87620e57a6ff46148fb550b7922bf2b4` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:37.178924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac28661-3ae5-4f99-9af3-87466c878b59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87620e57a6ff46148fb550b7922bf2b4)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:37.180879 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac28661-3ae5-4f99-9af3-87466c878b59 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:37.232220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bac28661-3ae5-4f99-9af3-87466c878b59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87620e57a6ff46148fb550b7922bf2b4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:37.241439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bac28661-3ae5-4f99-9af3-87466c878b59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87620e57a6ff46148fb550b7922bf2b4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:37.242091 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bac28661-3ae5-4f99-9af3-87466c878b59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87620e57a6ff46148fb550b7922bf2b4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:37.242781 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bac28661-3ae5-4f99-9af3-87466c878b59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87620e57a6ff46148fb550b7922bf2b4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:37.248248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:37.252334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bac28661-3ae5-4f99-9af3-87466c878b59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87620e57a6ff46148fb550b7922bf2b4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:37.271736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 964/1923] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:37 2022] DELETE /identity/v3/users/87620e57a6ff46148fb550b7922bf2b4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:37.610660 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d92ef-d315-43f2-872a-e756a0e8b85f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:37.613426 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f37d92ef-d315-43f2-872a-e756a0e8b85f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:37.636018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37d92ef-d315-43f2-872a-e756a0e8b85f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:37.729842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4d19a-c4e3-4d1a-9711-015c119b1537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:37.731252 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4d19a-c4e3-4d1a-9711-015c119b1537 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:37.731568 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4d19a-c4e3-4d1a-9711-015c119b1537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:37.731868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4d19a-c4e3-4d1a-9711-015c119b1537 admin admin] PATH_INFO: `/v3/projects/a9c2fd03d8054685920873b34856731c` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:37.733119 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4d19a-c4e3-4d1a-9711-015c119b1537 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9c2fd03d8054685920873b34856731c)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:37.735199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4d19a-c4e3-4d1a-9711-015c119b1537 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:37.793370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-06e4d19a-c4e3-4d1a-9711-015c119b1537 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9c2fd03d8054685920873b34856731c', 'request_id': 'req-06e4d19a-c4e3-4d1a-9711-015c119b1537'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:37.807599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-06e4d19a-c4e3-4d1a-9711-015c119b1537 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9c2fd03d8054685920873b34856731c', 'request_id': 'req-06e4d19a-c4e3-4d1a-9711-015c119b1537'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:37.813173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-06e4d19a-c4e3-4d1a-9711-015c119b1537 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9c2fd03d8054685920873b34856731c', 'request_id': 'req-06e4d19a-c4e3-4d1a-9711-015c119b1537'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:37.824695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-06e4d19a-c4e3-4d1a-9711-015c119b1537 admin admin] The token cache is being invalidate because project a9c2fd03d8054685920873b34856731c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:37.824977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-06e4d19a-c4e3-4d1a-9711-015c119b1537 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:37.825591 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-06e4d19a-c4e3-4d1a-9711-015c119b1537 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:37.827080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 960/1924] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:37 2022] DELETE /identity/v3/projects/a9c2fd03d8054685920873b34856731c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:37.832031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfa1b0-45dc-4316-9a19-ab753f8b3f28 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:37.832492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfa1b0-45dc-4316-9a19-ab753f8b3f28 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:37.832581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dfa1b0-45dc-4316-9a19-ab753f8b3f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:37.852058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-b6dfa1b0-45dc-4316-9a19-ab753f8b3f28 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:37.853248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b6dfa1b0-45dc-4316-9a19-ab753f8b3f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:37.914333 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 965/1925] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:37.923112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcc450-db43-4c9f-8b61-d69cc30cd391 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:37.923397 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcc450-db43-4c9f-8b61-d69cc30cd391 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:37.923755 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fcc450-db43-4c9f-8b61-d69cc30cd391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:37.948457 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-a6fcc450-db43-4c9f-8b61-d69cc30cd391 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:37.949987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a6fcc450-db43-4c9f-8b61-d69cc30cd391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:37.958963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 961/1926] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:37.963588 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfa1b0-45dc-4316-9a19-ab753f8b3f28 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:37.974339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dfa1b0-45dc-4316-9a19-ab753f8b3f28 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:37.981236 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9bf41-4c6a-40b2-b18d-aa362103048a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:37.981987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9bf41-4c6a-40b2-b18d-aa362103048a admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:37.982161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9bf41-4c6a-40b2-b18d-aa362103048a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:37.982394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab9bf41-4c6a-40b2-b18d-aa362103048a admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:37.983191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9bf41-4c6a-40b2-b18d-aa362103048a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:37.983624 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9bf41-4c6a-40b2-b18d-aa362103048a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:37.985302 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab9bf41-4c6a-40b2-b18d-aa362103048a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:38.002243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 966/1927] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:08:38.006991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcc450-db43-4c9f-8b61-d69cc30cd391 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:38.016931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fcc450-db43-4c9f-8b61-d69cc30cd391 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:38.025523 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dce24-99f3-47b4-a27a-52d222fa1044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:38.026159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dce24-99f3-47b4-a27a-52d222fa1044 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:38.026471 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dce24-99f3-47b4-a27a-52d222fa1044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:38.026737 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1dce24-99f3-47b4-a27a-52d222fa1044 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:38.027390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dce24-99f3-47b4-a27a-52d222fa1044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:38.029352 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1dce24-99f3-47b4-a27a-52d222fa1044 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:38.062262 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 962/1928] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:38 2022] POST /identity/v3/projects => generated 409 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:38.067030 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9bf41-4c6a-40b2-b18d-aa362103048a admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:38.079606 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab9bf41-4c6a-40b2-b18d-aa362103048a admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:38.151658 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef38ba-6312-4766-9224-4a0eefb14fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:38.152420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef38ba-6312-4766-9224-4a0eefb14fac admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:38.152808 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef38ba-6312-4766-9224-4a0eefb14fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:38.153084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef38ba-6312-4766-9224-4a0eefb14fac admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:38.153804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef38ba-6312-4766-9224-4a0eefb14fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:38.155875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef38ba-6312-4766-9224-4a0eefb14fac admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:38.189204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 967/1929] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:38 2022] POST /identity/v3/users => generated 305 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:38.193347 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dce24-99f3-47b4-a27a-52d222fa1044 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:38.204454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1dce24-99f3-47b4-a27a-52d222fa1044 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:38.209172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b4d56-eeaa-4b17-a6f8-2ab50e2485be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:38.209878 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b4d56-eeaa-4b17-a6f8-2ab50e2485be admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:38.210224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b4d56-eeaa-4b17-a6f8-2ab50e2485be admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:38.210560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b4d56-eeaa-4b17-a6f8-2ab50e2485be admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:38.211276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b4d56-eeaa-4b17-a6f8-2ab50e2485be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:38.212925 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b4d56-eeaa-4b17-a6f8-2ab50e2485be admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:38.224756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 963/1930] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:38.228959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef38ba-6312-4766-9224-4a0eefb14fac admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:38.239422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef38ba-6312-4766-9224-4a0eefb14fac admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:38.244539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359a9893-3af6-4359-98f3-86d92eebd40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:38.245231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a9893-3af6-4359-98f3-86d92eebd40d admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:38.245507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a9893-3af6-4359-98f3-86d92eebd40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:38.245779 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359a9893-3af6-4359-98f3-86d92eebd40d admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:38.246430 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a9893-3af6-4359-98f3-86d92eebd40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:38.248083 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359a9893-3af6-4359-98f3-86d92eebd40d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:38.259813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 968/1931] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:08:38.264580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b4d56-eeaa-4b17-a6f8-2ab50e2485be admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:38.275298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b4d56-eeaa-4b17-a6f8-2ab50e2485be admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:38.280176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04321f75-45b6-4354-93cf-641b1b0ddacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:38.280851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04321f75-45b6-4354-93cf-641b1b0ddacb admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:38.281282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04321f75-45b6-4354-93cf-641b1b0ddacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:38.281512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04321f75-45b6-4354-93cf-641b1b0ddacb admin admin] PATH_INFO: `/v3/projects/9d169589367643cf9574a2624acf0da8/users/fefc941411b24e95ac3e724ba9c38bea/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:38.296814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04321f75-45b6-4354-93cf-641b1b0ddacb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d169589367643cf9574a2624acf0da8, user_id=fefc941411b24e95ac3e724ba9c38bea, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:38.299566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04321f75-45b6-4354-93cf-641b1b0ddacb admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:38.302790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:38.316240 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 964/1932] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:38 2022] PUT /identity/v3/projects/9d169589367643cf9574a2624acf0da8/users/fefc941411b24e95ac3e724ba9c38bea/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:38.330674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8eceb-7bce-420b-b80d-346f2af18599 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:38.331054 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8eceb-7bce-420b-b80d-346f2af18599 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:38.331592 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8eceb-7bce-420b-b80d-346f2af18599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:38.365935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-dbb8eceb-7bce-420b-b80d-346f2af18599 None None] MFA Rules not processed for user `fefc941411b24e95ac3e724ba9c38bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:38.367200 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-dbb8eceb-7bce-420b-b80d-346f2af18599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:38.406548 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:38.453714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 969/1933] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:38 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:38.459604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04321f75-45b6-4354-93cf-641b1b0ddacb admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:38.586870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04321f75-45b6-4354-93cf-641b1b0ddacb admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:38.592962 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ceaf6-8ecd-4dd1-bbe9-3b3b46cdfe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:38.594174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ceaf6-8ecd-4dd1-bbe9-3b3b46cdfe0c admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:38.594174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ceaf6-8ecd-4dd1-bbe9-3b3b46cdfe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:38.594174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ceaf6-8ecd-4dd1-bbe9-3b3b46cdfe0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:38.595035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ceaf6-8ecd-4dd1-bbe9-3b3b46cdfe0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:38.597175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7ceaf6-8ecd-4dd1-bbe9-3b3b46cdfe0c admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:38.640102 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 965/1934] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:38 2022] POST /identity/v3/projects => generated 407 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:38.646749 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8eceb-7bce-420b-b80d-346f2af18599 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:38.662760 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8eceb-7bce-420b-b80d-346f2af18599 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:38.754354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bbe76-a862-41b1-ad65-ae15347b1527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:38.755267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bbe76-a862-41b1-ad65-ae15347b1527 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:38.755655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bbe76-a862-41b1-ad65-ae15347b1527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:38.755993 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bbe76-a862-41b1-ad65-ae15347b1527 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:38.757397 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bbe76-a862-41b1-ad65-ae15347b1527 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:38.764388 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564bbe76-a862-41b1-ad65-ae15347b1527 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:38.837766 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 970/1935] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:38 2022] POST /identity/v3/users => generated 310 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:38.842216 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ceaf6-8ecd-4dd1-bbe9-3b3b46cdfe0c admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:38.861170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ceaf6-8ecd-4dd1-bbe9-3b3b46cdfe0c admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:38.868455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e57426-ac9f-4a94-8fd2-718bc8ae1f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:38.869939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e57426-ac9f-4a94-8fd2-718bc8ae1f58 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:38.870491 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e57426-ac9f-4a94-8fd2-718bc8ae1f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:38.870858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e57426-ac9f-4a94-8fd2-718bc8ae1f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:38.872818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e57426-ac9f-4a94-8fd2-718bc8ae1f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:38.876859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e57426-ac9f-4a94-8fd2-718bc8ae1f58 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:38.897239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 966/1936] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:38.900706 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bbe76-a862-41b1-ad65-ae15347b1527 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:38.917010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bbe76-a862-41b1-ad65-ae15347b1527 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:38.927262 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc5129-66f1-47c0-8685-ccabb8567eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:38.928326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc5129-66f1-47c0-8685-ccabb8567eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:38.929165 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc5129-66f1-47c0-8685-ccabb8567eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:38.930235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc5129-66f1-47c0-8685-ccabb8567eaf admin admin] PATH_INFO: `/v3/projects/f962f19fc67a4c02836653ab4d091715/users/0a40cfaa464b4432952aa6a9642ec3ee/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:38.953561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc5129-66f1-47c0-8685-ccabb8567eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f962f19fc67a4c02836653ab4d091715, user_id=0a40cfaa464b4432952aa6a9642ec3ee, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:38.957680 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc5129-66f1-47c0-8685-ccabb8567eaf admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:38.982039 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 971/1937] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:38 2022] PUT /identity/v3/projects/f962f19fc67a4c02836653ab4d091715/users/0a40cfaa464b4432952aa6a9642ec3ee/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:38.986803 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e57426-ac9f-4a94-8fd2-718bc8ae1f58 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:39.014896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e57426-ac9f-4a94-8fd2-718bc8ae1f58 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:39.023681 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc5129-66f1-47c0-8685-ccabb8567eaf admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:39.026266 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-11dc5129-66f1-47c0-8685-ccabb8567eaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:39.133035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc5129-66f1-47c0-8685-ccabb8567eaf admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:39.138579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1680a1e-2e9a-4211-92a2-3efec567a579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:39.139339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1680a1e-2e9a-4211-92a2-3efec567a579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:39.139643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1680a1e-2e9a-4211-92a2-3efec567a579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:39.139916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1680a1e-2e9a-4211-92a2-3efec567a579 admin admin] PATH_INFO: `/v3/users/bd2298664242483cb083c85df077d351` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:39.141159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1680a1e-2e9a-4211-92a2-3efec567a579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd2298664242483cb083c85df077d351)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:39.143414 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1680a1e-2e9a-4211-92a2-3efec567a579 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:39.150197 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613e8b8-2a08-42fd-87df-d1093676677a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:39.163254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613e8b8-2a08-42fd-87df-d1093676677a admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:39.163254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613e8b8-2a08-42fd-87df-d1093676677a admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:39.163254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613e8b8-2a08-42fd-87df-d1093676677a admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:39.163771 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613e8b8-2a08-42fd-87df-d1093676677a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:39.175097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613e8b8-2a08-42fd-87df-d1093676677a admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:39.202930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 967/1938] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:39.203844 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613e8b8-2a08-42fd-87df-d1093676677a admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:39.216832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613e8b8-2a08-42fd-87df-d1093676677a admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:39.230397 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238aacc-ee28-4b0c-8bcb-0f6f2d7d9537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:39.231102 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238aacc-ee28-4b0c-8bcb-0f6f2d7d9537 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:39.231362 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:39.231491 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238aacc-ee28-4b0c-8bcb-0f6f2d7d9537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:39.231660 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238aacc-ee28-4b0c-8bcb-0f6f2d7d9537 admin admin] PATH_INFO: `/v3/projects/f962f19fc67a4c02836653ab4d091715/users/0a40cfaa464b4432952aa6a9642ec3ee/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:39.240206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238aacc-ee28-4b0c-8bcb-0f6f2d7d9537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f962f19fc67a4c02836653ab4d091715, user_id=0a40cfaa464b4432952aa6a9642ec3ee, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:39.242912 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238aacc-ee28-4b0c-8bcb-0f6f2d7d9537 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:39.244682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e1680a1e-2e9a-4211-92a2-3efec567a579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd2298664242483cb083c85df077d351'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:39.268695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e1680a1e-2e9a-4211-92a2-3efec567a579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd2298664242483cb083c85df077d351'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:39.269566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e1680a1e-2e9a-4211-92a2-3efec567a579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd2298664242483cb083c85df077d351'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:39.270432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e1680a1e-2e9a-4211-92a2-3efec567a579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd2298664242483cb083c85df077d351'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:39.273361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 968/1939] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:39 2022] PUT /identity/v3/projects/f962f19fc67a4c02836653ab4d091715/users/0a40cfaa464b4432952aa6a9642ec3ee/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:39.280221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6281133-9799-4083-8325-1abad14b6bd3 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:39.280508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6281133-9799-4083-8325-1abad14b6bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:39.280781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6281133-9799-4083-8325-1abad14b6bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:39.281646 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e1680a1e-2e9a-4211-92a2-3efec567a579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd2298664242483cb083c85df077d351'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:39.317288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 972/1940] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:39 2022] DELETE /identity/v3/users/bd2298664242483cb083c85df077d351 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:39.322369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5923238-111c-44c9-acd7-a0c9a96f4b3c None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:39.322583 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5923238-111c-44c9-acd7-a0c9a96f4b3c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:39.322859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5923238-111c-44c9-acd7-a0c9a96f4b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:39.337380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f6281133-9799-4083-8325-1abad14b6bd3 None None] MFA Rules not processed for user `0a40cfaa464b4432952aa6a9642ec3ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:39.339285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f6281133-9799-4083-8325-1abad14b6bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:39.344966 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-b5923238-111c-44c9-acd7-a0c9a96f4b3c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:39.346429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b5923238-111c-44c9-acd7-a0c9a96f4b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:39.432845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:39.450060 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 973/1941] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:39.461350 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5923238-111c-44c9-acd7-a0c9a96f4b3c None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:39.463166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b5923238-111c-44c9-acd7-a0c9a96f4b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:39.520565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 969/1942] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:39 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:39.535273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6281133-9799-4083-8325-1abad14b6bd3 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:39.543487 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f6281133-9799-4083-8325-1abad14b6bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:39.558979 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5923238-111c-44c9-acd7-a0c9a96f4b3c None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:39.566490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352637a-0e51-4dcb-9d61-8689f33908c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:39.567134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352637a-0e51-4dcb-9d61-8689f33908c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:39.567404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352637a-0e51-4dcb-9d61-8689f33908c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:39.569799 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6352637a-0e51-4dcb-9d61-8689f33908c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:39.593710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352637a-0e51-4dcb-9d61-8689f33908c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:39.598122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6352637a-0e51-4dcb-9d61-8689f33908c2 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:39.624293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 974/1943] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:08:39.626363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6281133-9799-4083-8325-1abad14b6bd3 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:39.630692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522e411-dd86-4e4f-ba48-5375a7a8df9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:39.631371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522e411-dd86-4e4f-ba48-5375a7a8df9b service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:39.631862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522e411-dd86-4e4f-ba48-5375a7a8df9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:39.632144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3522e411-dd86-4e4f-ba48-5375a7a8df9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:39.647573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522e411-dd86-4e4f-ba48-5375a7a8df9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:39.651292 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3522e411-dd86-4e4f-ba48-5375a7a8df9b service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:39.672174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 970/1944] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:39 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:39.958017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352637a-0e51-4dcb-9d61-8689f33908c2 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:39.959756 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6352637a-0e51-4dcb-9d61-8689f33908c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:39.978952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6352637a-0e51-4dcb-9d61-8689f33908c2 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:39.983500 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dce07f-8a7e-4a5e-8f7d-3607b8f8918e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:39.984109 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dce07f-8a7e-4a5e-8f7d-3607b8f8918e service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:39.984456 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dce07f-8a7e-4a5e-8f7d-3607b8f8918e service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:39.984989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dce07f-8a7e-4a5e-8f7d-3607b8f8918e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:39.987861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d6dce07f-8a7e-4a5e-8f7d-3607b8f8918e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:40.067101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dce07f-8a7e-4a5e-8f7d-3607b8f8918e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:40.068822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dce07f-8a7e-4a5e-8f7d-3607b8f8918e service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:40.072902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:40.081297 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 975/1945] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:39 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:40.208118 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522e411-dd86-4e4f-ba48-5375a7a8df9b service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:40.211782 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3522e411-dd86-4e4f-ba48-5375a7a8df9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:40.228767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3522e411-dd86-4e4f-ba48-5375a7a8df9b service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:40.237002 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1f029-a062-41a5-9353-f314dc474aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:40.237943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1f029-a062-41a5-9353-f314dc474aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:40.238227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1f029-a062-41a5-9353-f314dc474aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:40.238501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1f029-a062-41a5-9353-f314dc474aaf admin admin] PATH_INFO: `/v3/projects/b89bb001554444c5b200d9b6c0ea95e0` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:40.240082 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1f029-a062-41a5-9353-f314dc474aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b89bb001554444c5b200d9b6c0ea95e0)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:40.244820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1f029-a062-41a5-9353-f314dc474aaf admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:40.310121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-9bf1f029-a062-41a5-9353-f314dc474aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b89bb001554444c5b200d9b6c0ea95e0', 'request_id': 'req-9bf1f029-a062-41a5-9353-f314dc474aaf'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:40.321965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-9bf1f029-a062-41a5-9353-f314dc474aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b89bb001554444c5b200d9b6c0ea95e0', 'request_id': 'req-9bf1f029-a062-41a5-9353-f314dc474aaf'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:40.331515 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-9bf1f029-a062-41a5-9353-f314dc474aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b89bb001554444c5b200d9b6c0ea95e0', 'request_id': 'req-9bf1f029-a062-41a5-9353-f314dc474aaf'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:40.344452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-9bf1f029-a062-41a5-9353-f314dc474aaf admin admin] The token cache is being invalidate because project b89bb001554444c5b200d9b6c0ea95e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:40.345165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-9bf1f029-a062-41a5-9353-f314dc474aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:40.346088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-9bf1f029-a062-41a5-9353-f314dc474aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:40.348304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 971/1946] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:40 2022] DELETE /identity/v3/projects/b89bb001554444c5b200d9b6c0ea95e0 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:40.352659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dce07f-8a7e-4a5e-8f7d-3607b8f8918e service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:40.371780 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dce07f-8a7e-4a5e-8f7d-3607b8f8918e service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:40.472464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3eceb9-f95e-49e2-bec2-6e44183ff516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:40.474657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3eceb9-f95e-49e2-bec2-6e44183ff516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:40.474871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3eceb9-f95e-49e2-bec2-6e44183ff516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:40.475420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3eceb9-f95e-49e2-bec2-6e44183ff516 admin admin] PATH_INFO: `/v3/users/324bc0f422ed4f59a03ec9f0f0828f21` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:40.477744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3eceb9-f95e-49e2-bec2-6e44183ff516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=324bc0f422ed4f59a03ec9f0f0828f21)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:40.483669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3eceb9-f95e-49e2-bec2-6e44183ff516 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:40.589734 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1d3eceb9-f95e-49e2-bec2-6e44183ff516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324bc0f422ed4f59a03ec9f0f0828f21'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:40.607499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1d3eceb9-f95e-49e2-bec2-6e44183ff516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324bc0f422ed4f59a03ec9f0f0828f21'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:40.608760 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1d3eceb9-f95e-49e2-bec2-6e44183ff516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324bc0f422ed4f59a03ec9f0f0828f21'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:40.610102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1d3eceb9-f95e-49e2-bec2-6e44183ff516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324bc0f422ed4f59a03ec9f0f0828f21'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:40.630372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1d3eceb9-f95e-49e2-bec2-6e44183ff516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324bc0f422ed4f59a03ec9f0f0828f21'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:40.654722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 976/1947] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:40 2022] DELETE /identity/v3/users/324bc0f422ed4f59a03ec9f0f0828f21 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:40.681422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dea1a-510b-4148-8014-ba2f10480993 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:40.682026 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dea1a-510b-4148-8014-ba2f10480993 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:40.682026 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9dea1a-510b-4148-8014-ba2f10480993 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:40.684019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 972/1948] 10.222.0.27 () {54 vars in 921 bytes} [Thu Sep 8 16:08:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:08:40.717669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4e166-5d7c-4ad8-b387-8223edbdbe04 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:40.717975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4e166-5d7c-4ad8-b387-8223edbdbe04 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:40.718420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e4e166-5d7c-4ad8-b387-8223edbdbe04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:40.755229 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-d9e4e166-5d7c-4ad8-b387-8223edbdbe04 None None] MFA Rules not processed for user `6e9209e77ce44b128e25a8ddcf78f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:40.757179 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d9e4e166-5d7c-4ad8-b387-8223edbdbe04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:40.877706 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 977/1949] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Sep 8 16:08:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:41.059396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9dea1a-510b-4148-8014-ba2f10480993 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:41.061198 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-7b9dea1a-510b-4148-8014-ba2f10480993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:41.082751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9dea1a-510b-4148-8014-ba2f10480993 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:41.088792 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55036812-f229-4866-ba99-a2b18d65707d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:41.089417 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55036812-f229-4866-ba99-a2b18d65707d service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:41.089700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55036812-f229-4866-ba99-a2b18d65707d service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:41.089962 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55036812-f229-4866-ba99-a2b18d65707d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:41.100459 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55036812-f229-4866-ba99-a2b18d65707d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:41.102608 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55036812-f229-4866-ba99-a2b18d65707d service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:41.115026 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 973/1950] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:41.359399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4e166-5d7c-4ad8-b387-8223edbdbe04 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:41.360743 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d9e4e166-5d7c-4ad8-b387-8223edbdbe04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:41.378254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:41.460175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e4e166-5d7c-4ad8-b387-8223edbdbe04 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:41.465557 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544376c4-8753-4a25-8080-314150f9f82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:41.466536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544376c4-8753-4a25-8080-314150f9f82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:41.467054 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544376c4-8753-4a25-8080-314150f9f82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:41.467707 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544376c4-8753-4a25-8080-314150f9f82b admin admin] PATH_INFO: `/v3/projects/6f1b167073aa447f8f2dee124b733c81` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:41.470035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544376c4-8753-4a25-8080-314150f9f82b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f1b167073aa447f8f2dee124b733c81)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:41.472586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544376c4-8753-4a25-8080-314150f9f82b admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:41.540742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-544376c4-8753-4a25-8080-314150f9f82b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f1b167073aa447f8f2dee124b733c81', 'request_id': 'req-544376c4-8753-4a25-8080-314150f9f82b'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:41.550832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-544376c4-8753-4a25-8080-314150f9f82b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f1b167073aa447f8f2dee124b733c81', 'request_id': 'req-544376c4-8753-4a25-8080-314150f9f82b'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:41.560139 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-544376c4-8753-4a25-8080-314150f9f82b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f1b167073aa447f8f2dee124b733c81', 'request_id': 'req-544376c4-8753-4a25-8080-314150f9f82b'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:41.567469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55036812-f229-4866-ba99-a2b18d65707d service nova] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:41.568987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-55036812-f229-4866-ba99-a2b18d65707d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:41.585169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-544376c4-8753-4a25-8080-314150f9f82b admin admin] The token cache is being invalidate because project 6f1b167073aa447f8f2dee124b733c81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:41.585462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-544376c4-8753-4a25-8080-314150f9f82b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:41.586115 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-544376c4-8753-4a25-8080-314150f9f82b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:41.587781 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 978/1951] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:41 2022] DELETE /identity/v3/projects/6f1b167073aa447f8f2dee124b733c81 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:41.588448 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55036812-f229-4866-ba99-a2b18d65707d service nova] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:41.593183 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adfdd8-b140-422a-8b47-44830de7b208 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:41.593467 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adfdd8-b140-422a-8b47-44830de7b208 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:41.593790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77adfdd8-b140-422a-8b47-44830de7b208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:41.635351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-77adfdd8-b140-422a-8b47-44830de7b208 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:41.637790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-77adfdd8-b140-422a-8b47-44830de7b208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:41.647553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:41.667013 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010cd93-0a52-481b-9b33-fa3187aa8eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:41.668341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010cd93-0a52-481b-9b33-fa3187aa8eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:41.668923 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010cd93-0a52-481b-9b33-fa3187aa8eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:41.669592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010cd93-0a52-481b-9b33-fa3187aa8eb8 admin admin] PATH_INFO: `/v3/users/3db22bc465fa46a1a993e94599b1e3af` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:41.671258 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010cd93-0a52-481b-9b33-fa3187aa8eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3db22bc465fa46a1a993e94599b1e3af)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:41.675114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3010cd93-0a52-481b-9b33-fa3187aa8eb8 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:41.739943 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 979/1952] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:41.748331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34fceb-d199-448f-a56e-54362ca12061 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:41.748569 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34fceb-d199-448f-a56e-54362ca12061 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:41.748819 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34fceb-d199-448f-a56e-54362ca12061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:41.749988 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3010cd93-0a52-481b-9b33-fa3187aa8eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3db22bc465fa46a1a993e94599b1e3af'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:41.782082 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3010cd93-0a52-481b-9b33-fa3187aa8eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3db22bc465fa46a1a993e94599b1e3af'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:41.783007 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3010cd93-0a52-481b-9b33-fa3187aa8eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3db22bc465fa46a1a993e94599b1e3af'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:41.784406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3010cd93-0a52-481b-9b33-fa3187aa8eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3db22bc465fa46a1a993e94599b1e3af'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:41.784733 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-db34fceb-d199-448f-a56e-54362ca12061 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:41.786075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-db34fceb-d199-448f-a56e-54362ca12061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:41.809660 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3010cd93-0a52-481b-9b33-fa3187aa8eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3db22bc465fa46a1a993e94599b1e3af'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:41.811951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 980/1953] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:41.813324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34fceb-d199-448f-a56e-54362ca12061 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:41.832691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 974/1954] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:41 2022] DELETE /identity/v3/users/3db22bc465fa46a1a993e94599b1e3af => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:41.835268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34fceb-d199-448f-a56e-54362ca12061 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:41.838477 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610e7d6-5308-4a7f-99b0-8d745d209abf None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:41.839556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610e7d6-5308-4a7f-99b0-8d745d209abf None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:41.841730 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3610e7d6-5308-4a7f-99b0-8d745d209abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:41.872262 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-3610e7d6-5308-4a7f-99b0-8d745d209abf None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:41.874160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3610e7d6-5308-4a7f-99b0-8d745d209abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:41.924740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:41.970751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b139f9-6b1d-4202-b2f6-c040ac94c76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:41.971807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b139f9-6b1d-4202-b2f6-c040ac94c76f admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:41.972106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b139f9-6b1d-4202-b2f6-c040ac94c76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:41.972436 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b139f9-6b1d-4202-b2f6-c040ac94c76f admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:41.973614 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b139f9-6b1d-4202-b2f6-c040ac94c76f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:41.973921 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b139f9-6b1d-4202-b2f6-c040ac94c76f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:41.975701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b139f9-6b1d-4202-b2f6-c040ac94c76f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:41.994562 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 981/1955] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:08:41.999335 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b139f9-6b1d-4202-b2f6-c040ac94c76f admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:42.021209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 975/1956] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:42.038599 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b139f9-6b1d-4202-b2f6-c040ac94c76f admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:42.041258 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610e7d6-5308-4a7f-99b0-8d745d209abf None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:42.043835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3610e7d6-5308-4a7f-99b0-8d745d209abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:42.048586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd957d-31b3-4a43-9e6e-622f5e1e0ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:42.049245 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd957d-31b3-4a43-9e6e-622f5e1e0ac8 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:42.049532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd957d-31b3-4a43-9e6e-622f5e1e0ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:42.053088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd957d-31b3-4a43-9e6e-622f5e1e0ac8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:42.053768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd957d-31b3-4a43-9e6e-622f5e1e0ac8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:42.060389 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd957d-31b3-4a43-9e6e-622f5e1e0ac8 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:42.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3610e7d6-5308-4a7f-99b0-8d745d209abf None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:42.158822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 982/1957] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:41 2022] POST /identity/v3/projects => generated 381 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:42.166957 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd957d-31b3-4a43-9e6e-622f5e1e0ac8 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:42.170123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cd667-09ea-403d-a2c3-afe6e5e386cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:42.176214 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cd667-09ea-403d-a2c3-afe6e5e386cf service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:42.176533 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cd667-09ea-403d-a2c3-afe6e5e386cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:42.177210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cd667-09ea-403d-a2c3-afe6e5e386cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:42.201428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cd667-09ea-403d-a2c3-afe6e5e386cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:42.203049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cd667-09ea-403d-a2c3-afe6e5e386cf service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:42.218615 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 976/1958] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:42.308599 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd957d-31b3-4a43-9e6e-622f5e1e0ac8 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:42.314048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc2ad2-cc93-48e6-a50c-9c9d15b66e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:42.315212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc2ad2-cc93-48e6-a50c-9c9d15b66e6d admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:42.315212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc2ad2-cc93-48e6-a50c-9c9d15b66e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:42.315212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc2ad2-cc93-48e6-a50c-9c9d15b66e6d admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:42.315960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc2ad2-cc93-48e6-a50c-9c9d15b66e6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:42.318873 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc2ad2-cc93-48e6-a50c-9c9d15b66e6d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:42.350966 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 983/1959] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:42 2022] POST /identity/v3/users => generated 291 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:42.354982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cd667-09ea-403d-a2c3-afe6e5e386cf service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:42.367045 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cd667-09ea-403d-a2c3-afe6e5e386cf service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:42.372494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acbd35-9f0a-4892-8a42-4e70f6117b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:42.373117 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acbd35-9f0a-4892-8a42-4e70f6117b7a admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:42.373393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acbd35-9f0a-4892-8a42-4e70f6117b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:42.373873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3acbd35-9f0a-4892-8a42-4e70f6117b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:42.374540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3acbd35-9f0a-4892-8a42-4e70f6117b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:42.376421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3acbd35-9f0a-4892-8a42-4e70f6117b7a admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:42.390360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 977/1960] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:08:42.395072 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc2ad2-cc93-48e6-a50c-9c9d15b66e6d admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:42.411605 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc2ad2-cc93-48e6-a50c-9c9d15b66e6d admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:42.418673 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c80096-b661-48af-9ff1-39123714952d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:42.420011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c80096-b661-48af-9ff1-39123714952d admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:42.420295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c80096-b661-48af-9ff1-39123714952d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:42.420590 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c80096-b661-48af-9ff1-39123714952d admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:42.421328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c80096-b661-48af-9ff1-39123714952d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:42.423064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c80096-b661-48af-9ff1-39123714952d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:42.447739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 984/1961] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:42.449844 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acbd35-9f0a-4892-8a42-4e70f6117b7a admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:42.455122 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:42.462879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3acbd35-9f0a-4892-8a42-4e70f6117b7a admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:42.475548 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5e0ac-6a49-48a6-af0f-0bf3c3bd8aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:42.476571 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5e0ac-6a49-48a6-af0f-0bf3c3bd8aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:42.477230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5e0ac-6a49-48a6-af0f-0bf3c3bd8aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:42.478333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa5e0ac-6a49-48a6-af0f-0bf3c3bd8aa7 admin admin] PATH_INFO: `/v3/projects/a604c38c7a0646d9a2d1187d7e5005d4/users/c530a88e55e94133bf754342e4a10105/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:42.503032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5e0ac-6a49-48a6-af0f-0bf3c3bd8aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a604c38c7a0646d9a2d1187d7e5005d4, user_id=c530a88e55e94133bf754342e4a10105, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:42.505967 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa5e0ac-6a49-48a6-af0f-0bf3c3bd8aa7 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:42.534811 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 978/1962] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:42 2022] PUT /identity/v3/projects/a604c38c7a0646d9a2d1187d7e5005d4/users/c530a88e55e94133bf754342e4a10105/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:42.560642 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20177781-e83a-4d7a-8248-b9ed4ffcb97e None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:42.560934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20177781-e83a-4d7a-8248-b9ed4ffcb97e None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:42.561275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20177781-e83a-4d7a-8248-b9ed4ffcb97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:42.601219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-20177781-e83a-4d7a-8248-b9ed4ffcb97e None None] MFA Rules not processed for user `c530a88e55e94133bf754342e4a10105`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:42.602670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-20177781-e83a-4d7a-8248-b9ed4ffcb97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:42.701541 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 985/1963] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:42.703825 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5e0ac-6a49-48a6-af0f-0bf3c3bd8aa7 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:42.739057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20177781-e83a-4d7a-8248-b9ed4ffcb97e None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:42.740816 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-20177781-e83a-4d7a-8248-b9ed4ffcb97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:42.755702 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:42.776799 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa5e0ac-6a49-48a6-af0f-0bf3c3bd8aa7 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:42.781605 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7987e23-280f-402a-b214-6144ff46acc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:42.782303 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7987e23-280f-402a-b214-6144ff46acc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:42.782642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7987e23-280f-402a-b214-6144ff46acc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:42.782979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7987e23-280f-402a-b214-6144ff46acc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:42.783708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7987e23-280f-402a-b214-6144ff46acc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:42.785682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7987e23-280f-402a-b214-6144ff46acc8 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:42.809458 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20177781-e83a-4d7a-8248-b9ed4ffcb97e None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:42.812950 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 979/1964] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:42 2022] POST /identity/v3/projects => generated 381 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:08:42.814102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf88f8bb-c8b8-4419-9497-dda56a6a84ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:42.814731 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88f8bb-c8b8-4419-9497-dda56a6a84ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:42.815006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88f8bb-c8b8-4419-9497-dda56a6a84ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:42.815277 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf88f8bb-c8b8-4419-9497-dda56a6a84ce admin admin] PATH_INFO: `/v3/projects/5c011058726b4756b3178dbf5037739c` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:42.816464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88f8bb-c8b8-4419-9497-dda56a6a84ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c011058726b4756b3178dbf5037739c)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:42.817067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7987e23-280f-402a-b214-6144ff46acc8 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:42.818399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf88f8bb-c8b8-4419-9497-dda56a6a84ce admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:42.869858 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bf88f8bb-c8b8-4419-9497-dda56a6a84ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c011058726b4756b3178dbf5037739c', 'request_id': 'req-bf88f8bb-c8b8-4419-9497-dda56a6a84ce'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:42.878655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bf88f8bb-c8b8-4419-9497-dda56a6a84ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c011058726b4756b3178dbf5037739c', 'request_id': 'req-bf88f8bb-c8b8-4419-9497-dda56a6a84ce'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:42.881014 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7987e23-280f-402a-b214-6144ff46acc8 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:42.885201 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bf88f8bb-c8b8-4419-9497-dda56a6a84ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c011058726b4756b3178dbf5037739c', 'request_id': 'req-bf88f8bb-c8b8-4419-9497-dda56a6a84ce'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:42.896997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bf88f8bb-c8b8-4419-9497-dda56a6a84ce admin admin] The token cache is being invalidate because project 5c011058726b4756b3178dbf5037739c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:42.897279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bf88f8bb-c8b8-4419-9497-dda56a6a84ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:42.897967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bf88f8bb-c8b8-4419-9497-dda56a6a84ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:42.899769 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 986/1965] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:42 2022] DELETE /identity/v3/projects/5c011058726b4756b3178dbf5037739c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:42.905191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912186c9-e96c-4b2b-9caa-5bfb9fa30607 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:42.905470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912186c9-e96c-4b2b-9caa-5bfb9fa30607 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:42.905738 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912186c9-e96c-4b2b-9caa-5bfb9fa30607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:42.931350 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-912186c9-e96c-4b2b-9caa-5bfb9fa30607 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:42.932846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-912186c9-e96c-4b2b-9caa-5bfb9fa30607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:42.946832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe86de-f226-49f1-b995-450618d7c5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:42.947377 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe86de-f226-49f1-b995-450618d7c5a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:42.947688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe86de-f226-49f1-b995-450618d7c5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:42.947981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe86de-f226-49f1-b995-450618d7c5a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:42.948617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe86de-f226-49f1-b995-450618d7c5a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:42.950538 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe86de-f226-49f1-b995-450618d7c5a2 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:42.968371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:42.974978 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 980/1966] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:42 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:08:42.978191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe86de-f226-49f1-b995-450618d7c5a2 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:42.980005 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-00fe86de-f226-49f1-b995-450618d7c5a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:42.991480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe86de-f226-49f1-b995-450618d7c5a2 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:42.995226 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 987/1967] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:42.996049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1aba1-b4f5-44d5-9e2c-d1d1f7a97c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:42.996649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1aba1-b4f5-44d5-9e2c-d1d1f7a97c02 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:42.996923 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1aba1-b4f5-44d5-9e2c-d1d1f7a97c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:42.997190 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc1aba1-b4f5-44d5-9e2c-d1d1f7a97c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:42.997829 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1aba1-b4f5-44d5-9e2c-d1d1f7a97c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:42.999424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc1aba1-b4f5-44d5-9e2c-d1d1f7a97c02 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:43.003303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb76b2-552c-4e94-bb87-98a15bcde15a None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:43.003617 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb76b2-552c-4e94-bb87-98a15bcde15a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:43.003884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb76b2-552c-4e94-bb87-98a15bcde15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:43.009525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 981/1968] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:43.013503 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1aba1-b4f5-44d5-9e2c-d1d1f7a97c02 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:43.023080 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc1aba1-b4f5-44d5-9e2c-d1d1f7a97c02 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:43.024865 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-7cfb76b2-552c-4e94-bb87-98a15bcde15a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:43.026040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-7cfb76b2-552c-4e94-bb87-98a15bcde15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:43.028280 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adadc02-a647-434f-a609-fcff44ada645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:43.029036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adadc02-a647-434f-a609-fcff44ada645 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:43.029426 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adadc02-a647-434f-a609-fcff44ada645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:43.029699 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adadc02-a647-434f-a609-fcff44ada645 admin admin] PATH_INFO: `/v3/projects/6a95127f01384f1fb48e6c5d9b1846a0/users/433e4137cba94004880d7a4144e6837c/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:43.034946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 988/1969] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:43.038998 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb76b2-552c-4e94-bb87-98a15bcde15a None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:43.045378 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adadc02-a647-434f-a609-fcff44ada645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a95127f01384f1fb48e6c5d9b1846a0, user_id=433e4137cba94004880d7a4144e6837c, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:43.047880 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb76b2-552c-4e94-bb87-98a15bcde15a None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:43.048274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adadc02-a647-434f-a609-fcff44ada645 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:43.052391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32a424-8060-4609-8de3-75c2536bf115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:43.053016 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32a424-8060-4609-8de3-75c2536bf115 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:43.053358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32a424-8060-4609-8de3-75c2536bf115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:43.053627 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32a424-8060-4609-8de3-75c2536bf115 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:43.054431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32a424-8060-4609-8de3-75c2536bf115 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:43.054797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32a424-8060-4609-8de3-75c2536bf115 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:43.056532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32a424-8060-4609-8de3-75c2536bf115 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:43.062259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 982/1970] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:43 2022] PUT /identity/v3/projects/6a95127f01384f1fb48e6c5d9b1846a0/users/433e4137cba94004880d7a4144e6837c/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:43.066536 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adadc02-a647-434f-a609-fcff44ada645 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:43.073543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 989/1971] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:08:43.074719 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32a424-8060-4609-8de3-75c2536bf115 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:43.085829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32a424-8060-4609-8de3-75c2536bf115 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:43.109594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:43.137827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adadc02-a647-434f-a609-fcff44ada645 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:43.143677 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd887e-7988-4839-a37c-b5d3923d6862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:43.144431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd887e-7988-4839-a37c-b5d3923d6862 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:43.144776 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd887e-7988-4839-a37c-b5d3923d6862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:43.145134 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd887e-7988-4839-a37c-b5d3923d6862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:43.145867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd887e-7988-4839-a37c-b5d3923d6862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:43.147576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd887e-7988-4839-a37c-b5d3923d6862 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:43.150683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b07d7a-f8a9-4573-a537-fc32030dba91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:43.151520 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b07d7a-f8a9-4573-a537-fc32030dba91 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:43.151884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b07d7a-f8a9-4573-a537-fc32030dba91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:43.152219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b07d7a-f8a9-4573-a537-fc32030dba91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:43.152922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b07d7a-f8a9-4573-a537-fc32030dba91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:43.154877 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b07d7a-f8a9-4573-a537-fc32030dba91 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:43.163856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 983/1972] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:08:43.169552 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd887e-7988-4839-a37c-b5d3923d6862 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:43.182809 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd887e-7988-4839-a37c-b5d3923d6862 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:43.189283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695cb4f-8ace-4702-96de-4e63b2c8f632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:43.190136 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695cb4f-8ace-4702-96de-4e63b2c8f632 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:43.190798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695cb4f-8ace-4702-96de-4e63b2c8f632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:43.191223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695cb4f-8ace-4702-96de-4e63b2c8f632 admin admin] PATH_INFO: `/v3/projects/6a95127f01384f1fb48e6c5d9b1846a0/users/433e4137cba94004880d7a4144e6837c/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:43.193943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695cb4f-8ace-4702-96de-4e63b2c8f632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a95127f01384f1fb48e6c5d9b1846a0, user_id=433e4137cba94004880d7a4144e6837c, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:43.196915 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 990/1973] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:43 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:43.197768 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695cb4f-8ace-4702-96de-4e63b2c8f632 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:43.201600 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b07d7a-f8a9-4573-a537-fc32030dba91 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:43.216901 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 984/1974] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:43 2022] PUT /identity/v3/projects/6a95127f01384f1fb48e6c5d9b1846a0/users/433e4137cba94004880d7a4144e6837c/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:43.227611 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3b8ae-9be5-4ea4-9bc5-3192599a418d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:43.227919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3b8ae-9be5-4ea4-9bc5-3192599a418d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:43.228270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3b8ae-9be5-4ea4-9bc5-3192599a418d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:43.260807 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f7e3b8ae-9be5-4ea4-9bc5-3192599a418d None None] MFA Rules not processed for user `433e4137cba94004880d7a4144e6837c`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:43.262179 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f7e3b8ae-9be5-4ea4-9bc5-3192599a418d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:43.305967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b07d7a-f8a9-4573-a537-fc32030dba91 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:43.349661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 985/1975] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:43.361681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3b8ae-9be5-4ea4-9bc5-3192599a418d None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:43.363510 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f7e3b8ae-9be5-4ea4-9bc5-3192599a418d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:43.389464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883fcac-381e-4808-a3a5-ea8d00d1474a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:43.390121 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883fcac-381e-4808-a3a5-ea8d00d1474a admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:43.390401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883fcac-381e-4808-a3a5-ea8d00d1474a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:43.390672 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883fcac-381e-4808-a3a5-ea8d00d1474a admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:43.391874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883fcac-381e-4808-a3a5-ea8d00d1474a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:43.393338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9883fcac-381e-4808-a3a5-ea8d00d1474a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:43.406084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:43.424268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 991/1976] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:43 2022] POST /identity/v3/users => generated 291 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:08:43.431847 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883fcac-381e-4808-a3a5-ea8d00d1474a admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:43.434247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3b8ae-9be5-4ea4-9bc5-3192599a418d None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:43.440622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc262e-8637-46d2-b1f4-4f0eecf435bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:43.441636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc262e-8637-46d2-b1f4-4f0eecf435bd service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:43.442179 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc262e-8637-46d2-b1f4-4f0eecf435bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:43.442791 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc262e-8637-46d2-b1f4-4f0eecf435bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:43.444549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883fcac-381e-4808-a3a5-ea8d00d1474a admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:43.450253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb01b16-1d6b-4e74-b8dd-d820624996ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:43.450912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb01b16-1d6b-4e74-b8dd-d820624996ef admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:43.451198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb01b16-1d6b-4e74-b8dd-d820624996ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:43.451490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb01b16-1d6b-4e74-b8dd-d820624996ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:43.452154 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb01b16-1d6b-4e74-b8dd-d820624996ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:43.453918 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb01b16-1d6b-4e74-b8dd-d820624996ef admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:43.455973 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc262e-8637-46d2-b1f4-4f0eecf435bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:43.458672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc262e-8637-46d2-b1f4-4f0eecf435bd service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:43.466354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 992/1977] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:08:43.470635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb01b16-1d6b-4e74-b8dd-d820624996ef admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:43.471964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 986/1978] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:43 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:08:43.481937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb01b16-1d6b-4e74-b8dd-d820624996ef admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:43.487696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1d178-4485-438d-8955-8b9285947fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:43.488342 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1d178-4485-438d-8955-8b9285947fec admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:43.488625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1d178-4485-438d-8955-8b9285947fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:43.488900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1d178-4485-438d-8955-8b9285947fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:43.489734 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1d178-4485-438d-8955-8b9285947fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:43.491988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1d178-4485-438d-8955-8b9285947fec admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:43.507555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 993/1979] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:08:43.508761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc262e-8637-46d2-b1f4-4f0eecf435bd service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:43.519582 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc262e-8637-46d2-b1f4-4f0eecf435bd service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:43.525898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7908e-c236-461b-992c-5a4fefab956d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:43.526645 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7908e-c236-461b-992c-5a4fefab956d admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:43.526925 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7908e-c236-461b-992c-5a4fefab956d admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:43.527191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7908e-c236-461b-992c-5a4fefab956d admin admin] PATH_INFO: `/v3/projects/bafdc72abf95401f8784411bc8e55822/users/2127487c50884141b2776f05e96471fd/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:43.547910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7908e-c236-461b-992c-5a4fefab956d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bafdc72abf95401f8784411bc8e55822, user_id=2127487c50884141b2776f05e96471fd, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:43.552069 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7908e-c236-461b-992c-5a4fefab956d admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:43.578694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 987/1980] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:43 2022] PUT /identity/v3/projects/bafdc72abf95401f8784411bc8e55822/users/2127487c50884141b2776f05e96471fd/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:43.584376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d5b6e-5c2c-4951-9040-8c1f465b53c1 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:43.584644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d5b6e-5c2c-4951-9040-8c1f465b53c1 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:43.584940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d5b6e-5c2c-4951-9040-8c1f465b53c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:43.609555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-805d5b6e-5c2c-4951-9040-8c1f465b53c1 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:43.610909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-805d5b6e-5c2c-4951-9040-8c1f465b53c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:43.615310 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7908e-c236-461b-992c-5a4fefab956d admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:43.672284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:43.752022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7908e-c236-461b-992c-5a4fefab956d admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:43.756799 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743decc-cd4f-4836-82f7-1116de232f32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:43.757663 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743decc-cd4f-4836-82f7-1116de232f32 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:43.758022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743decc-cd4f-4836-82f7-1116de232f32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:43.758533 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743decc-cd4f-4836-82f7-1116de232f32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:43.760929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5743decc-cd4f-4836-82f7-1116de232f32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:43.819464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 994/1981] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:43.832946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d5b6e-5c2c-4951-9040-8c1f465b53c1 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:43.834561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-805d5b6e-5c2c-4951-9040-8c1f465b53c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:43.842826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743decc-cd4f-4836-82f7-1116de232f32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:43.844349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743decc-cd4f-4836-82f7-1116de232f32 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:43.853402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 988/1982] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:43.919252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d5b6e-5c2c-4951-9040-8c1f465b53c1 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:43.924126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea57d7f-54e3-417d-add9-2da6b55311ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:43.924813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea57d7f-54e3-417d-add9-2da6b55311ba service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:43.925170 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea57d7f-54e3-417d-add9-2da6b55311ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:43.925451 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea57d7f-54e3-417d-add9-2da6b55311ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:43.936467 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea57d7f-54e3-417d-add9-2da6b55311ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:43.938087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea57d7f-54e3-417d-add9-2da6b55311ba service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:43.949852 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 995/1983] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:44.170587 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743decc-cd4f-4836-82f7-1116de232f32 service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:44.188710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743decc-cd4f-4836-82f7-1116de232f32 service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:44.193377 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c6e5f-c2c7-4840-b5e5-56f76fcc12cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:44.194112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c6e5f-c2c7-4840-b5e5-56f76fcc12cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:44.194440 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c6e5f-c2c7-4840-b5e5-56f76fcc12cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:44.194722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c6e5f-c2c7-4840-b5e5-56f76fcc12cf admin admin] PATH_INFO: `/v3/users/c530a88e55e94133bf754342e4a10105` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:44.196290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c6e5f-c2c7-4840-b5e5-56f76fcc12cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c530a88e55e94133bf754342e4a10105)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:44.199369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40c6e5f-c2c7-4840-b5e5-56f76fcc12cf admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:44.255898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a40c6e5f-c2c7-4840-b5e5-56f76fcc12cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c530a88e55e94133bf754342e4a10105'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:44.268805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a40c6e5f-c2c7-4840-b5e5-56f76fcc12cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c530a88e55e94133bf754342e4a10105'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:44.269434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a40c6e5f-c2c7-4840-b5e5-56f76fcc12cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c530a88e55e94133bf754342e4a10105'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:44.270268 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a40c6e5f-c2c7-4840-b5e5-56f76fcc12cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c530a88e55e94133bf754342e4a10105'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:44.276883 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:44.286990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a40c6e5f-c2c7-4840-b5e5-56f76fcc12cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c530a88e55e94133bf754342e4a10105'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:44.302321 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 989/1984] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:44 2022] DELETE /identity/v3/users/c530a88e55e94133bf754342e4a10105 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:44.307781 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327878e-4e42-4a14-8b44-c26cf7e75fd8 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:44.308130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327878e-4e42-4a14-8b44-c26cf7e75fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:44.308350 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327878e-4e42-4a14-8b44-c26cf7e75fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:44.331740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-9327878e-4e42-4a14-8b44-c26cf7e75fd8 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:44.332978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9327878e-4e42-4a14-8b44-c26cf7e75fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:44.400991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 996/1985] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:44.411906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c6e5f-c2c7-4840-b5e5-56f76fcc12cf admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:44.413664 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a40c6e5f-c2c7-4840-b5e5-56f76fcc12cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:44.470616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c6e5f-c2c7-4840-b5e5-56f76fcc12cf admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:44.474314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc9432-d224-467d-917c-bc1b6251012e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:44.474929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc9432-d224-467d-917c-bc1b6251012e service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:44.475209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc9432-d224-467d-917c-bc1b6251012e service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:44.475498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc9432-d224-467d-917c-bc1b6251012e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:44.486372 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc9432-d224-467d-917c-bc1b6251012e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:44.487877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc9432-d224-467d-917c-bc1b6251012e service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:44.498676 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 990/1986] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:44.891035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327878e-4e42-4a14-8b44-c26cf7e75fd8 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:44.893327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9327878e-4e42-4a14-8b44-c26cf7e75fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:44.914233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327878e-4e42-4a14-8b44-c26cf7e75fd8 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:44.940806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d234d-ec37-404b-8e1a-cea2b7368deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:44.941478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d234d-ec37-404b-8e1a-cea2b7368deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:44.941751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d234d-ec37-404b-8e1a-cea2b7368deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:44.942019 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d234d-ec37-404b-8e1a-cea2b7368deb admin admin] PATH_INFO: `/v3/projects/a604c38c7a0646d9a2d1187d7e5005d4` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:44.946355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d234d-ec37-404b-8e1a-cea2b7368deb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a604c38c7a0646d9a2d1187d7e5005d4)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:44.953573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d234d-ec37-404b-8e1a-cea2b7368deb admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:44.977184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334439e-5c43-40b3-91ab-04cfddd96cef None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:44.978092 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334439e-5c43-40b3-91ab-04cfddd96cef None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:44.978544 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2334439e-5c43-40b3-91ab-04cfddd96cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:45.012058 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-2334439e-5c43-40b3-91ab-04cfddd96cef None None] MFA Rules not processed for user `2127487c50884141b2776f05e96471fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:45.013789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2334439e-5c43-40b3-91ab-04cfddd96cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:45.080977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5a2d234d-ec37-404b-8e1a-cea2b7368deb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a604c38c7a0646d9a2d1187d7e5005d4', 'request_id': 'req-5a2d234d-ec37-404b-8e1a-cea2b7368deb'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:45.095904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5a2d234d-ec37-404b-8e1a-cea2b7368deb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a604c38c7a0646d9a2d1187d7e5005d4', 'request_id': 'req-5a2d234d-ec37-404b-8e1a-cea2b7368deb'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:45.105194 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:45.107971 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5a2d234d-ec37-404b-8e1a-cea2b7368deb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a604c38c7a0646d9a2d1187d7e5005d4', 'request_id': 'req-5a2d234d-ec37-404b-8e1a-cea2b7368deb'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:45.151487 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 991/1987] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:45.152988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5a2d234d-ec37-404b-8e1a-cea2b7368deb admin admin] The token cache is being invalidate because project a604c38c7a0646d9a2d1187d7e5005d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:45.153445 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5a2d234d-ec37-404b-8e1a-cea2b7368deb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:45.154912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5a2d234d-ec37-404b-8e1a-cea2b7368deb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:45.161518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334439e-5c43-40b3-91ab-04cfddd96cef None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:45.163431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2334439e-5c43-40b3-91ab-04cfddd96cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:45.173594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 997/1988] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:44 2022] DELETE /identity/v3/projects/a604c38c7a0646d9a2d1187d7e5005d4 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:45.190833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d234d-ec37-404b-8e1a-cea2b7368deb admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:45.193749 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5a2d234d-ec37-404b-8e1a-cea2b7368deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:45.248153 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2334439e-5c43-40b3-91ab-04cfddd96cef None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:45.278997 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:45.289107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d234d-ec37-404b-8e1a-cea2b7368deb admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:45.295108 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af709b16-ce3c-472c-9b44-1cefccfff71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:45.296039 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af709b16-ce3c-472c-9b44-1cefccfff71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:45.296924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af709b16-ce3c-472c-9b44-1cefccfff71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:45.297265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af709b16-ce3c-472c-9b44-1cefccfff71e admin admin] PATH_INFO: `/v3/users/433e4137cba94004880d7a4144e6837c` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:45.299039 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af709b16-ce3c-472c-9b44-1cefccfff71e admin admin] RBAC: Authorizing `identity:delete_user(user_id=433e4137cba94004880d7a4144e6837c)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:45.301452 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af709b16-ce3c-472c-9b44-1cefccfff71e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:45.309497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca8320-017e-4edd-a887-b4ef6e7592ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:45.310300 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca8320-017e-4edd-a887-b4ef6e7592ba service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:45.310645 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca8320-017e-4edd-a887-b4ef6e7592ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:45.310995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca8320-017e-4edd-a887-b4ef6e7592ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:45.312950 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6bca8320-017e-4edd-a887-b4ef6e7592ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:45.358319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-af709b16-ce3c-472c-9b44-1cefccfff71e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '433e4137cba94004880d7a4144e6837c'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:45.366327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-af709b16-ce3c-472c-9b44-1cefccfff71e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '433e4137cba94004880d7a4144e6837c'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:45.367055 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-af709b16-ce3c-472c-9b44-1cefccfff71e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '433e4137cba94004880d7a4144e6837c'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:45.367770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-af709b16-ce3c-472c-9b44-1cefccfff71e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '433e4137cba94004880d7a4144e6837c'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:45.375346 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-af709b16-ce3c-472c-9b44-1cefccfff71e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '433e4137cba94004880d7a4144e6837c'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:45.387418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca8320-017e-4edd-a887-b4ef6e7592ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:45.388992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca8320-017e-4edd-a887-b4ef6e7592ba service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:45.394812 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 998/1989] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:45 2022] DELETE /identity/v3/users/433e4137cba94004880d7a4144e6837c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:45.398525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 992/1990] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:08:45.733118 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af709b16-ce3c-472c-9b44-1cefccfff71e admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:45.735729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-af709b16-ce3c-472c-9b44-1cefccfff71e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:45.858388 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af709b16-ce3c-472c-9b44-1cefccfff71e admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:45.872032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5b2b1d-2ba3-44fe-9e38-0ec4224397f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:45.872693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b2b1d-2ba3-44fe-9e38-0ec4224397f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:45.872973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b2b1d-2ba3-44fe-9e38-0ec4224397f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:45.873251 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5b2b1d-2ba3-44fe-9e38-0ec4224397f9 admin admin] PATH_INFO: `/v3/projects/6a95127f01384f1fb48e6c5d9b1846a0` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:45.874395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b2b1d-2ba3-44fe-9e38-0ec4224397f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a95127f01384f1fb48e6c5d9b1846a0)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:45.876388 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5b2b1d-2ba3-44fe-9e38-0ec4224397f9 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:45.889024 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:45.936508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8e5b2b1d-2ba3-44fe-9e38-0ec4224397f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a95127f01384f1fb48e6c5d9b1846a0', 'request_id': 'req-8e5b2b1d-2ba3-44fe-9e38-0ec4224397f9'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:45.947071 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8e5b2b1d-2ba3-44fe-9e38-0ec4224397f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a95127f01384f1fb48e6c5d9b1846a0', 'request_id': 'req-8e5b2b1d-2ba3-44fe-9e38-0ec4224397f9'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:45.958206 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8e5b2b1d-2ba3-44fe-9e38-0ec4224397f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a95127f01384f1fb48e6c5d9b1846a0', 'request_id': 'req-8e5b2b1d-2ba3-44fe-9e38-0ec4224397f9'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:45.970902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8e5b2b1d-2ba3-44fe-9e38-0ec4224397f9 admin admin] The token cache is being invalidate because project 6a95127f01384f1fb48e6c5d9b1846a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:45.971342 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8e5b2b1d-2ba3-44fe-9e38-0ec4224397f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:45.972041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8e5b2b1d-2ba3-44fe-9e38-0ec4224397f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:45.975470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 999/1991] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:45 2022] DELETE /identity/v3/projects/6a95127f01384f1fb48e6c5d9b1846a0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:45.979877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac3193-ded8-4dc6-a2fa-2ca1c78d0e4c None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:45.980359 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac3193-ded8-4dc6-a2fa-2ca1c78d0e4c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:45.980608 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac3193-ded8-4dc6-a2fa-2ca1c78d0e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:46.002665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-b6ac3193-ded8-4dc6-a2fa-2ca1c78d0e4c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:46.004743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b6ac3193-ded8-4dc6-a2fa-2ca1c78d0e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:46.095718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 993/1992] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:46.099309 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bcc0c-5416-4891-a659-8ea19e33bfdf None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:46.099674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bcc0c-5416-4891-a659-8ea19e33bfdf None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:46.099944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60bcc0c-5416-4891-a659-8ea19e33bfdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:46.121700 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-f60bcc0c-5416-4891-a659-8ea19e33bfdf None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:46.122824 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f60bcc0c-5416-4891-a659-8ea19e33bfdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:46.134893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1000/1993] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:46.135597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac3193-ded8-4dc6-a2fa-2ca1c78d0e4c None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:46.153005 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac3193-ded8-4dc6-a2fa-2ca1c78d0e4c None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:46.157808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6db97-28d8-4bb4-969f-c677daddfdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:46.158436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6db97-28d8-4bb4-969f-c677daddfdaf admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:46.158711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6db97-28d8-4bb4-969f-c677daddfdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:46.158986 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6db97-28d8-4bb4-969f-c677daddfdaf admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:46.159729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6db97-28d8-4bb4-969f-c677daddfdaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:46.160042 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6db97-28d8-4bb4-969f-c677daddfdaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:46.171630 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6db97-28d8-4bb4-969f-c677daddfdaf admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:46.178332 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 994/1994] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:08:46.180626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bcc0c-5416-4891-a659-8ea19e33bfdf None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:46.197718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60bcc0c-5416-4891-a659-8ea19e33bfdf None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:46.213425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae75cf-04c6-4789-9db8-6e6b520c63eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:46.214336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae75cf-04c6-4789-9db8-6e6b520c63eb admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:46.214613 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae75cf-04c6-4789-9db8-6e6b520c63eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:46.214889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae75cf-04c6-4789-9db8-6e6b520c63eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:46.215598 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae75cf-04c6-4789-9db8-6e6b520c63eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:46.217578 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ae75cf-04c6-4789-9db8-6e6b520c63eb admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:46.273116 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1001/1995] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:46 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:46.277532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6db97-28d8-4bb4-969f-c677daddfdaf admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:46.294314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6db97-28d8-4bb4-969f-c677daddfdaf admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:46.381710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a2d32-2204-4ad1-a94c-0e146fce9b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:46.382632 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a2d32-2204-4ad1-a94c-0e146fce9b9b admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:46.382943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a2d32-2204-4ad1-a94c-0e146fce9b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:46.383606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a2d32-2204-4ad1-a94c-0e146fce9b9b admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:46.383933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a2d32-2204-4ad1-a94c-0e146fce9b9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:46.386567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a2d32-2204-4ad1-a94c-0e146fce9b9b admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:46.418145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:46.430809 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 995/1996] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:46 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:46.431884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae75cf-04c6-4789-9db8-6e6b520c63eb admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:46.442625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae75cf-04c6-4789-9db8-6e6b520c63eb admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:46.447706 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd0a29-57dc-4edb-a504-d5707584c21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:46.448385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd0a29-57dc-4edb-a504-d5707584c21c admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:46.448661 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd0a29-57dc-4edb-a504-d5707584c21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:46.448930 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd0a29-57dc-4edb-a504-d5707584c21c admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:46.449626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd0a29-57dc-4edb-a504-d5707584c21c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:46.451490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd0a29-57dc-4edb-a504-d5707584c21c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:46.463510 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1002/1997] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:46.465406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a2d32-2204-4ad1-a94c-0e146fce9b9b admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:46.475286 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a2d32-2204-4ad1-a94c-0e146fce9b9b admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:46.480107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f409b9-be49-4d27-99d9-03861b6ee3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:46.480783 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f409b9-be49-4d27-99d9-03861b6ee3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:46.481068 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f409b9-be49-4d27-99d9-03861b6ee3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:46.481338 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f409b9-be49-4d27-99d9-03861b6ee3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:46.481982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f409b9-be49-4d27-99d9-03861b6ee3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:46.483617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f409b9-be49-4d27-99d9-03861b6ee3c8 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:46.494995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 996/1998] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:46.499298 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd0a29-57dc-4edb-a504-d5707584c21c admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:46.509256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd0a29-57dc-4edb-a504-d5707584c21c admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:46.514232 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bd2c6-5b93-44b2-92c7-3adcd2e326c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:46.514905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bd2c6-5b93-44b2-92c7-3adcd2e326c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:46.515170 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bd2c6-5b93-44b2-92c7-3adcd2e326c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:46.515479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bd2c6-5b93-44b2-92c7-3adcd2e326c4 admin admin] PATH_INFO: `/v3/projects/80193aab2e754a2a8a40d9e401606d0f/users/7bf6f3e1012b4ec082a3a4596911e982/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:46.530560 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bd2c6-5b93-44b2-92c7-3adcd2e326c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80193aab2e754a2a8a40d9e401606d0f, user_id=7bf6f3e1012b4ec082a3a4596911e982, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:46.533351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bd2c6-5b93-44b2-92c7-3adcd2e326c4 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:46.547093 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1003/1999] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:46 2022] PUT /identity/v3/projects/80193aab2e754a2a8a40d9e401606d0f/users/7bf6f3e1012b4ec082a3a4596911e982/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:46.558631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b712994-1448-4507-9c6f-f7a22527499a None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:46.558914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b712994-1448-4507-9c6f-f7a22527499a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:46.559188 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b712994-1448-4507-9c6f-f7a22527499a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:46.587592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-6b712994-1448-4507-9c6f-f7a22527499a None None] MFA Rules not processed for user `7bf6f3e1012b4ec082a3a4596911e982`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:46.588994 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6b712994-1448-4507-9c6f-f7a22527499a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:46.656901 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 997/2000] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:46.670134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bd2c6-5b93-44b2-92c7-3adcd2e326c4 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:46.672134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-634bd2c6-5b93-44b2-92c7-3adcd2e326c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:46.731562 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bd2c6-5b93-44b2-92c7-3adcd2e326c4 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:46.735301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da94c5-7aa3-41bb-b2e0-c39ff87d70e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:46.735951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da94c5-7aa3-41bb-b2e0-c39ff87d70e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:46.736223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da94c5-7aa3-41bb-b2e0-c39ff87d70e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:46.736490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da94c5-7aa3-41bb-b2e0-c39ff87d70e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:46.745605 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da94c5-7aa3-41bb-b2e0-c39ff87d70e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:46.747032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da94c5-7aa3-41bb-b2e0-c39ff87d70e7 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:46.755658 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1004/2001] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:46.874879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670e59e-f149-4749-9af2-dd2693759714 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:46.875636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670e59e-f149-4749-9af2-dd2693759714 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:46.875912 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670e59e-f149-4749-9af2-dd2693759714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:46.897112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-e670e59e-f149-4749-9af2-dd2693759714 None None] MFA Rules not processed for user `2127487c50884141b2776f05e96471fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:46.898566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e670e59e-f149-4749-9af2-dd2693759714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:46.956505 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 998/2002] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:51.330507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da94c5-7aa3-41bb-b2e0-c39ff87d70e7 service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:51.332069 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a8da94c5-7aa3-41bb-b2e0-c39ff87d70e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:51.387278 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:51.413896 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da94c5-7aa3-41bb-b2e0-c39ff87d70e7 service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:51.420047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8142b9ce-dea0-4fa1-887d-2aa799ec09d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:51.420726 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142b9ce-dea0-4fa1-887d-2aa799ec09d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:51.420984 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142b9ce-dea0-4fa1-887d-2aa799ec09d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:51.421248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8142b9ce-dea0-4fa1-887d-2aa799ec09d5 admin admin] PATH_INFO: `/v3/users/fefc941411b24e95ac3e724ba9c38bea` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:51.422513 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142b9ce-dea0-4fa1-887d-2aa799ec09d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fefc941411b24e95ac3e724ba9c38bea)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:51.424518 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8142b9ce-dea0-4fa1-887d-2aa799ec09d5 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:51.489186 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8142b9ce-dea0-4fa1-887d-2aa799ec09d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fefc941411b24e95ac3e724ba9c38bea'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:51.502104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8142b9ce-dea0-4fa1-887d-2aa799ec09d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fefc941411b24e95ac3e724ba9c38bea'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:51.503174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8142b9ce-dea0-4fa1-887d-2aa799ec09d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fefc941411b24e95ac3e724ba9c38bea'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:51.504728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8142b9ce-dea0-4fa1-887d-2aa799ec09d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fefc941411b24e95ac3e724ba9c38bea'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:51.516010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8142b9ce-dea0-4fa1-887d-2aa799ec09d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fefc941411b24e95ac3e724ba9c38bea'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:51.525288 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670e59e-f149-4749-9af2-dd2693759714 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:51.540775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670e59e-f149-4749-9af2-dd2693759714 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:51.551953 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b1a11-c86d-4370-967e-e17addb9abe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:51.552625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b1a11-c86d-4370-967e-e17addb9abe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:51.552906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b1a11-c86d-4370-967e-e17addb9abe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:51.553182 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b1a11-c86d-4370-967e-e17addb9abe8 admin admin] PATH_INFO: `/v3/users/7bf6f3e1012b4ec082a3a4596911e982` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:51.555179 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b1a11-c86d-4370-967e-e17addb9abe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bf6f3e1012b4ec082a3a4596911e982)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:51.557160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b1a11-c86d-4370-967e-e17addb9abe8 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:51.558660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1005/2003] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:51 2022] DELETE /identity/v3/users/fefc941411b24e95ac3e724ba9c38bea => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:51.565171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13adc87d-70ff-4e53-aec9-476ebef93729 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:51.565452 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13adc87d-70ff-4e53-aec9-476ebef93729 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:51.565853 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13adc87d-70ff-4e53-aec9-476ebef93729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:51.596801 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-13adc87d-70ff-4e53-aec9-476ebef93729 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:51.600576 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-13adc87d-70ff-4e53-aec9-476ebef93729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:51.627263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-315b1a11-c86d-4370-967e-e17addb9abe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bf6f3e1012b4ec082a3a4596911e982'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:51.635230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-315b1a11-c86d-4370-967e-e17addb9abe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bf6f3e1012b4ec082a3a4596911e982'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:51.636249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-315b1a11-c86d-4370-967e-e17addb9abe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bf6f3e1012b4ec082a3a4596911e982'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:51.637222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-315b1a11-c86d-4370-967e-e17addb9abe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bf6f3e1012b4ec082a3a4596911e982'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:51.661373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-315b1a11-c86d-4370-967e-e17addb9abe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bf6f3e1012b4ec082a3a4596911e982'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:51.667329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:51.679550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 999/2004] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:51 2022] DELETE /identity/v3/users/7bf6f3e1012b4ec082a3a4596911e982 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:51.683418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126e6ee-d0be-4b0f-a647-a59f70f60732 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:51.683978 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126e6ee-d0be-4b0f-a647-a59f70f60732 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:51.684259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126e6ee-d0be-4b0f-a647-a59f70f60732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:51.690338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1006/2005] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:51.702231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13adc87d-70ff-4e53-aec9-476ebef93729 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:51.704865 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-13adc87d-70ff-4e53-aec9-476ebef93729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:51.718404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-5126e6ee-d0be-4b0f-a647-a59f70f60732 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:51.719703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5126e6ee-d0be-4b0f-a647-a59f70f60732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:51.810810 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1000/2006] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:51.815661 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13adc87d-70ff-4e53-aec9-476ebef93729 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:51.822868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126e6ee-d0be-4b0f-a647-a59f70f60732 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:51.833106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:51.886136 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe90e40-e368-4587-a870-8f3eacfe3d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:51.886782 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe90e40-e368-4587-a870-8f3eacfe3d40 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:51.887162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe90e40-e368-4587-a870-8f3eacfe3d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:51.887455 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe90e40-e368-4587-a870-8f3eacfe3d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:51.900476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe90e40-e368-4587-a870-8f3eacfe3d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:51.902917 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe90e40-e368-4587-a870-8f3eacfe3d40 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:51.917136 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1007/2007] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:51.967786 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126e6ee-d0be-4b0f-a647-a59f70f60732 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:51.972716 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abf944-9fe4-4985-8b87-09d92cd0e869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:51.973506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abf944-9fe4-4985-8b87-09d92cd0e869 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:51.973923 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abf944-9fe4-4985-8b87-09d92cd0e869 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:51.974335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abf944-9fe4-4985-8b87-09d92cd0e869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:51.988721 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abf944-9fe4-4985-8b87-09d92cd0e869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:51.990525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abf944-9fe4-4985-8b87-09d92cd0e869 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:52.034853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1001/2008] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:52.204098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe90e40-e368-4587-a870-8f3eacfe3d40 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:52.205795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4fe90e40-e368-4587-a870-8f3eacfe3d40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:52.222379 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe90e40-e368-4587-a870-8f3eacfe3d40 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:52.227106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9af372-fdd9-4e88-a860-03bf3ca3ec0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:52.227874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9af372-fdd9-4e88-a860-03bf3ca3ec0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:52.228235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9af372-fdd9-4e88-a860-03bf3ca3ec0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:52.228549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9af372-fdd9-4e88-a860-03bf3ca3ec0d admin admin] PATH_INFO: `/v3/users/917b548bcbc44174931775eaf6999d60` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:52.229850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9af372-fdd9-4e88-a860-03bf3ca3ec0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=917b548bcbc44174931775eaf6999d60)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:52.232473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9af372-fdd9-4e88-a860-03bf3ca3ec0d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:52.327405 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6f9af372-fdd9-4e88-a860-03bf3ca3ec0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '917b548bcbc44174931775eaf6999d60'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:52.340999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6f9af372-fdd9-4e88-a860-03bf3ca3ec0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '917b548bcbc44174931775eaf6999d60'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:52.342485 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6f9af372-fdd9-4e88-a860-03bf3ca3ec0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '917b548bcbc44174931775eaf6999d60'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:52.343290 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6f9af372-fdd9-4e88-a860-03bf3ca3ec0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '917b548bcbc44174931775eaf6999d60'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:52.353909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6f9af372-fdd9-4e88-a860-03bf3ca3ec0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '917b548bcbc44174931775eaf6999d60'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:52.379872 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1008/2009] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:52 2022] DELETE /identity/v3/users/917b548bcbc44174931775eaf6999d60 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:52.384696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3919a-159e-4f36-bbf3-4e91e1c9624d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:52.385292 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3919a-159e-4f36-bbf3-4e91e1c9624d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:52.385960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f3919a-159e-4f36-bbf3-4e91e1c9624d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:52.411171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-e7f3919a-159e-4f36-bbf3-4e91e1c9624d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:52.412889 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e7f3919a-159e-4f36-bbf3-4e91e1c9624d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:52.486081 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9af372-fdd9-4e88-a860-03bf3ca3ec0d admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:52.488241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6f9af372-fdd9-4e88-a860-03bf3ca3ec0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:52.538534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1002/2010] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:52.582571 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3919a-159e-4f36-bbf3-4e91e1c9624d None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:52.584763 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e7f3919a-159e-4f36-bbf3-4e91e1c9624d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:52.627988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9af372-fdd9-4e88-a860-03bf3ca3ec0d admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:52.635772 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91237684-0dfa-46d7-86c9-c852d06f1295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:52.636539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91237684-0dfa-46d7-86c9-c852d06f1295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:52.636934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91237684-0dfa-46d7-86c9-c852d06f1295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:52.637276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91237684-0dfa-46d7-86c9-c852d06f1295 admin admin] PATH_INFO: `/v3/projects/9d169589367643cf9574a2624acf0da8` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:52.638564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91237684-0dfa-46d7-86c9-c852d06f1295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d169589367643cf9574a2624acf0da8)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:52.641831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91237684-0dfa-46d7-86c9-c852d06f1295 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:52.659741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:52.681920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f3919a-159e-4f36-bbf3-4e91e1c9624d None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:52.685931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395e090-b0df-488e-b159-21805fcbc543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:52.687213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395e090-b0df-488e-b159-21805fcbc543 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:52.687535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395e090-b0df-488e-b159-21805fcbc543 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:52.687820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395e090-b0df-488e-b159-21805fcbc543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:52.696262 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:52.701159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395e090-b0df-488e-b159-21805fcbc543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:52.702904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395e090-b0df-488e-b159-21805fcbc543 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:52.715572 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1003/2011] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:08:52.718885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-91237684-0dfa-46d7-86c9-c852d06f1295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d169589367643cf9574a2624acf0da8', 'request_id': 'req-91237684-0dfa-46d7-86c9-c852d06f1295'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:52.729604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-91237684-0dfa-46d7-86c9-c852d06f1295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d169589367643cf9574a2624acf0da8', 'request_id': 'req-91237684-0dfa-46d7-86c9-c852d06f1295'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:52.737946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-91237684-0dfa-46d7-86c9-c852d06f1295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d169589367643cf9574a2624acf0da8', 'request_id': 'req-91237684-0dfa-46d7-86c9-c852d06f1295'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:52.753609 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-91237684-0dfa-46d7-86c9-c852d06f1295 admin admin] The token cache is being invalidate because project 9d169589367643cf9574a2624acf0da8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:52.753952 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-91237684-0dfa-46d7-86c9-c852d06f1295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:52.754962 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-91237684-0dfa-46d7-86c9-c852d06f1295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:52.759540 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1009/2012] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:52 2022] DELETE /identity/v3/projects/9d169589367643cf9574a2624acf0da8 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:52.761457 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395e090-b0df-488e-b159-21805fcbc543 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:52.777868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395e090-b0df-488e-b159-21805fcbc543 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:52.854387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb01aa-f813-4578-aa92-d9fd1ca765c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:52.855536 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb01aa-f813-4578-aa92-d9fd1ca765c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:52.855536 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb01aa-f813-4578-aa92-d9fd1ca765c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:52.855763 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb01aa-f813-4578-aa92-d9fd1ca765c9 admin admin] PATH_INFO: `/v3/users/0a40cfaa464b4432952aa6a9642ec3ee` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:52.857849 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb01aa-f813-4578-aa92-d9fd1ca765c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a40cfaa464b4432952aa6a9642ec3ee)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:52.859116 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb01aa-f813-4578-aa92-d9fd1ca765c9 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:52.926280 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3ffb01aa-f813-4578-aa92-d9fd1ca765c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a40cfaa464b4432952aa6a9642ec3ee'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:52.936685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3ffb01aa-f813-4578-aa92-d9fd1ca765c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a40cfaa464b4432952aa6a9642ec3ee'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:52.937431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3ffb01aa-f813-4578-aa92-d9fd1ca765c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a40cfaa464b4432952aa6a9642ec3ee'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:52.940140 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3ffb01aa-f813-4578-aa92-d9fd1ca765c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a40cfaa464b4432952aa6a9642ec3ee'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:52.956023 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3ffb01aa-f813-4578-aa92-d9fd1ca765c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a40cfaa464b4432952aa6a9642ec3ee'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:52.977284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1004/2013] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:52 2022] DELETE /identity/v3/users/0a40cfaa464b4432952aa6a9642ec3ee => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:52.990626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91237684-0dfa-46d7-86c9-c852d06f1295 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:52.992041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-91237684-0dfa-46d7-86c9-c852d06f1295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:53.007506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91237684-0dfa-46d7-86c9-c852d06f1295 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:53.084531 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb01aa-f813-4578-aa92-d9fd1ca765c9 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:53.086446 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3ffb01aa-f813-4578-aa92-d9fd1ca765c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:53.171746 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c30cc-3490-488b-a213-67757b860e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:53.172639 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c30cc-3490-488b-a213-67757b860e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:53.172985 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c30cc-3490-488b-a213-67757b860e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:53.173395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c30cc-3490-488b-a213-67757b860e1a admin admin] PATH_INFO: `/v3/projects/80193aab2e754a2a8a40d9e401606d0f` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:53.174910 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c30cc-3490-488b-a213-67757b860e1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=80193aab2e754a2a8a40d9e401606d0f)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:53.177897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0c30cc-3490-488b-a213-67757b860e1a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:53.256452 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb01aa-f813-4578-aa92-d9fd1ca765c9 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:53.261919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066bee66-9f7c-462f-8aaa-a1d37399bd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:53.262886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066bee66-9f7c-462f-8aaa-a1d37399bd48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:53.263298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066bee66-9f7c-462f-8aaa-a1d37399bd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:53.263615 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066bee66-9f7c-462f-8aaa-a1d37399bd48 admin admin] PATH_INFO: `/v3/projects/1f291db8b4964794928b05f3a19f053f` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:53.264896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066bee66-9f7c-462f-8aaa-a1d37399bd48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f291db8b4964794928b05f3a19f053f)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:53.267013 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066bee66-9f7c-462f-8aaa-a1d37399bd48 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:53.268802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:53.329785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-7c0c30cc-3490-488b-a213-67757b860e1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80193aab2e754a2a8a40d9e401606d0f', 'request_id': 'req-7c0c30cc-3490-488b-a213-67757b860e1a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.338941 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-7c0c30cc-3490-488b-a213-67757b860e1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80193aab2e754a2a8a40d9e401606d0f', 'request_id': 'req-7c0c30cc-3490-488b-a213-67757b860e1a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.349350 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-066bee66-9f7c-462f-8aaa-a1d37399bd48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f291db8b4964794928b05f3a19f053f', 'request_id': 'req-066bee66-9f7c-462f-8aaa-a1d37399bd48'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.353724 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-7c0c30cc-3490-488b-a213-67757b860e1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80193aab2e754a2a8a40d9e401606d0f', 'request_id': 'req-7c0c30cc-3490-488b-a213-67757b860e1a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.372847 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-066bee66-9f7c-462f-8aaa-a1d37399bd48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f291db8b4964794928b05f3a19f053f', 'request_id': 'req-066bee66-9f7c-462f-8aaa-a1d37399bd48'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.384778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-7c0c30cc-3490-488b-a213-67757b860e1a admin admin] The token cache is being invalidate because project 80193aab2e754a2a8a40d9e401606d0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:53.385184 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-7c0c30cc-3490-488b-a213-67757b860e1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.386514 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-7c0c30cc-3490-488b-a213-67757b860e1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.389126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1010/2014] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:52 2022] DELETE /identity/v3/projects/80193aab2e754a2a8a40d9e401606d0f => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:53.389436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-066bee66-9f7c-462f-8aaa-a1d37399bd48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f291db8b4964794928b05f3a19f053f', 'request_id': 'req-066bee66-9f7c-462f-8aaa-a1d37399bd48'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.396001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff08c54-3645-46bd-842f-5611b45000c3 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:53.396285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff08c54-3645-46bd-842f-5611b45000c3 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:53.396561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff08c54-3645-46bd-842f-5611b45000c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:53.403417 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-066bee66-9f7c-462f-8aaa-a1d37399bd48 admin admin] The token cache is being invalidate because project 1f291db8b4964794928b05f3a19f053f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:53.403753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-066bee66-9f7c-462f-8aaa-a1d37399bd48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.404409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-066bee66-9f7c-462f-8aaa-a1d37399bd48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.406125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1005/2015] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:53 2022] DELETE /identity/v3/projects/1f291db8b4964794928b05f3a19f053f => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:53.409842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066bee66-9f7c-462f-8aaa-a1d37399bd48 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:53.411232 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-066bee66-9f7c-462f-8aaa-a1d37399bd48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:53.431375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-4ff08c54-3645-46bd-842f-5611b45000c3 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:53.433619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4ff08c54-3645-46bd-842f-5611b45000c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:53.493038 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066bee66-9f7c-462f-8aaa-a1d37399bd48 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:53.498652 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bf0cc-5f1c-42b7-b267-c823dc5e818e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:53.499432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bf0cc-5f1c-42b7-b267-c823dc5e818e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:53.499845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bf0cc-5f1c-42b7-b267-c823dc5e818e admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:53.500208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bf0cc-5f1c-42b7-b267-c823dc5e818e admin admin] PATH_INFO: `/v3/users/204b90d5623c4d8e80e7748c0883f142` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:53.501827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bf0cc-5f1c-42b7-b267-c823dc5e818e admin admin] RBAC: Authorizing `identity:delete_user(user_id=204b90d5623c4d8e80e7748c0883f142)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:53.504610 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:53.504880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bf0cc-5f1c-42b7-b267-c823dc5e818e admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:53.535652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1011/2016] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:53.537593 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c13aed-34ca-4184-b194-8f0ab176b815 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:53.538283 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c13aed-34ca-4184-b194-8f0ab176b815 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:53.538664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c13aed-34ca-4184-b194-8f0ab176b815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:53.568862 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-d0c13aed-34ca-4184-b194-8f0ab176b815 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:53.570367 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d0c13aed-34ca-4184-b194-8f0ab176b815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:53.588422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cd3bf0cc-5f1c-42b7-b267-c823dc5e818e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '204b90d5623c4d8e80e7748c0883f142'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.591649 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1012/2017] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:53.595149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c13aed-34ca-4184-b194-8f0ab176b815 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:53.605804 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cd3bf0cc-5f1c-42b7-b267-c823dc5e818e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '204b90d5623c4d8e80e7748c0883f142'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.606978 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cd3bf0cc-5f1c-42b7-b267-c823dc5e818e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '204b90d5623c4d8e80e7748c0883f142'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.607617 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c13aed-34ca-4184-b194-8f0ab176b815 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:53.608011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cd3bf0cc-5f1c-42b7-b267-c823dc5e818e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '204b90d5623c4d8e80e7748c0883f142'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.612646 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e854042-d531-4267-9490-4bda034dab9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:53.613280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e854042-d531-4267-9490-4bda034dab9e admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:53.613556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e854042-d531-4267-9490-4bda034dab9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:53.613841 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e854042-d531-4267-9490-4bda034dab9e admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:53.614911 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e854042-d531-4267-9490-4bda034dab9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:53.614911 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e854042-d531-4267-9490-4bda034dab9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:53.616977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e854042-d531-4267-9490-4bda034dab9e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:53.632034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cd3bf0cc-5f1c-42b7-b267-c823dc5e818e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '204b90d5623c4d8e80e7748c0883f142'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.633615 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1013/2018] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:08:53.634580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e854042-d531-4267-9490-4bda034dab9e admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:53.638439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0e854042-d531-4267-9490-4bda034dab9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:53.663350 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1006/2019] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:53 2022] DELETE /identity/v3/users/204b90d5623c4d8e80e7748c0883f142 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:53.667947 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bf0cc-5f1c-42b7-b267-c823dc5e818e admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:53.669742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-cd3bf0cc-5f1c-42b7-b267-c823dc5e818e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:53.673356 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e854042-d531-4267-9490-4bda034dab9e admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:53.713923 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:53.806222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bf0cc-5f1c-42b7-b267-c823dc5e818e admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:53.812077 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b6be3-6e9e-4ca9-9a17-73064a8e2a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:53.812497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b6be3-6e9e-4ca9-9a17-73064a8e2a7f admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:53.812985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b6be3-6e9e-4ca9-9a17-73064a8e2a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:53.813262 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b6be3-6e9e-4ca9-9a17-73064a8e2a7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:53.814324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b6be3-6e9e-4ca9-9a17-73064a8e2a7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:53.816750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b6be3-6e9e-4ca9-9a17-73064a8e2a7f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:53.821891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110b93f-6427-48e4-92b7-9b47aed6aeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:53.822624 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110b93f-6427-48e4-92b7-9b47aed6aeb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:53.823070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110b93f-6427-48e4-92b7-9b47aed6aeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:53.823408 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110b93f-6427-48e4-92b7-9b47aed6aeb7 admin admin] PATH_INFO: `/v3/projects/f962f19fc67a4c02836653ab4d091715` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:53.824740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110b93f-6427-48e4-92b7-9b47aed6aeb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f962f19fc67a4c02836653ab4d091715)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:53.826740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110b93f-6427-48e4-92b7-9b47aed6aeb7 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:53.849938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1007/2020] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:53 2022] POST /identity/v3/projects => generated 371 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:53.854040 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b6be3-6e9e-4ca9-9a17-73064a8e2a7f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:53.885441 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1110b93f-6427-48e4-92b7-9b47aed6aeb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f962f19fc67a4c02836653ab4d091715', 'request_id': 'req-1110b93f-6427-48e4-92b7-9b47aed6aeb7'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.897663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1110b93f-6427-48e4-92b7-9b47aed6aeb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f962f19fc67a4c02836653ab4d091715', 'request_id': 'req-1110b93f-6427-48e4-92b7-9b47aed6aeb7'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.904876 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1110b93f-6427-48e4-92b7-9b47aed6aeb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f962f19fc67a4c02836653ab4d091715', 'request_id': 'req-1110b93f-6427-48e4-92b7-9b47aed6aeb7'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.918658 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1110b93f-6427-48e4-92b7-9b47aed6aeb7 admin admin] The token cache is being invalidate because project f962f19fc67a4c02836653ab4d091715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:53.918954 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1110b93f-6427-48e4-92b7-9b47aed6aeb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.919878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1110b93f-6427-48e4-92b7-9b47aed6aeb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:53.922144 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1014/2021] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:53 2022] DELETE /identity/v3/projects/f962f19fc67a4c02836653ab4d091715 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:53.927049 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8bd3f-764b-4d51-a834-eab18e15cf98 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:53.927339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8bd3f-764b-4d51-a834-eab18e15cf98 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:53.927649 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8bd3f-764b-4d51-a834-eab18e15cf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:53.948658 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b6be3-6e9e-4ca9-9a17-73064a8e2a7f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:53.950683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-23b8bd3f-764b-4d51-a834-eab18e15cf98 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:53.952301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-23b8bd3f-764b-4d51-a834-eab18e15cf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:54.024302 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19791fc9-7aa0-40d0-8c9f-2ffaf6d32a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:54.025073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19791fc9-7aa0-40d0-8c9f-2ffaf6d32a09 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.025434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19791fc9-7aa0-40d0-8c9f-2ffaf6d32a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.025847 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19791fc9-7aa0-40d0-8c9f-2ffaf6d32a09 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.026694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19791fc9-7aa0-40d0-8c9f-2ffaf6d32a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:54.029064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19791fc9-7aa0-40d0-8c9f-2ffaf6d32a09 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:54.045290 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1015/2022] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:54.055297 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0f4f6-ed34-400f-bb9c-81b41648143b None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.056160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0f4f6-ed34-400f-bb9c-81b41648143b None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.056160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c0f4f6-ed34-400f-bb9c-81b41648143b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.061247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1008/2023] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:53 2022] POST /identity/v3/users => generated 286 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:08:54.071635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19791fc9-7aa0-40d0-8c9f-2ffaf6d32a09 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:54.071858 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:54.075357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-19791fc9-7aa0-40d0-8c9f-2ffaf6d32a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:54.095926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-64c0f4f6-ed34-400f-bb9c-81b41648143b None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:54.098157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-64c0f4f6-ed34-400f-bb9c-81b41648143b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:54.099092 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19791fc9-7aa0-40d0-8c9f-2ffaf6d32a09 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:54.106425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12530e4f-87e6-4863-bc60-22c321292412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:54.107247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12530e4f-87e6-4863-bc60-22c321292412 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.107548 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12530e4f-87e6-4863-bc60-22c321292412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.107929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12530e4f-87e6-4863-bc60-22c321292412 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.108675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12530e4f-87e6-4863-bc60-22c321292412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:54.112521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12530e4f-87e6-4863-bc60-22c321292412 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:54.122208 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1016/2024] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:54.123223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0f4f6-ed34-400f-bb9c-81b41648143b None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:54.126061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-64c0f4f6-ed34-400f-bb9c-81b41648143b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:54.131271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1009/2025] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:54.131271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12530e4f-87e6-4863-bc60-22c321292412 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:54.140439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c0f4f6-ed34-400f-bb9c-81b41648143b None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:54.145663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6fbee-a793-4fa8-a79a-af5e1d32f100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:54.146887 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6fbee-a793-4fa8-a79a-af5e1d32f100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.147161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6fbee-a793-4fa8-a79a-af5e1d32f100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.147432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6fbee-a793-4fa8-a79a-af5e1d32f100 admin admin] PATH_INFO: `/v3/projects/221f816e91c04414b1fbb27dacc0bb41` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.148738 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6fbee-a793-4fa8-a79a-af5e1d32f100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=221f816e91c04414b1fbb27dacc0bb41)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:54.150843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6fbee-a793-4fa8-a79a-af5e1d32f100 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:54.152067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12530e4f-87e6-4863-bc60-22c321292412 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:54.156999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19796b-163a-4324-9c0b-d18987fe96ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:54.157655 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19796b-163a-4324-9c0b-d18987fe96ae admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.157973 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19796b-163a-4324-9c0b-d18987fe96ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.158942 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a19796b-163a-4324-9c0b-d18987fe96ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.159602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19796b-163a-4324-9c0b-d18987fe96ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:54.160064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19796b-163a-4324-9c0b-d18987fe96ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:54.161688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a19796b-163a-4324-9c0b-d18987fe96ae admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:54.209330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1010/2026] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:08:54.210125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19796b-163a-4324-9c0b-d18987fe96ae admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:54.232607 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a19796b-163a-4324-9c0b-d18987fe96ae admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:54.241472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee755f2-ad8f-429a-bfb6-ef4fafa8ba1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:54.242215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee755f2-ad8f-429a-bfb6-ef4fafa8ba1f admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.242495 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee755f2-ad8f-429a-bfb6-ef4fafa8ba1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.242763 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee755f2-ad8f-429a-bfb6-ef4fafa8ba1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.243808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee755f2-ad8f-429a-bfb6-ef4fafa8ba1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:54.246236 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee755f2-ad8f-429a-bfb6-ef4fafa8ba1f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:54.249441 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:54.265631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1011/2027] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:54.269113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee755f2-ad8f-429a-bfb6-ef4fafa8ba1f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:54.277822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-34e6fbee-a793-4fa8-a79a-af5e1d32f100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '221f816e91c04414b1fbb27dacc0bb41', 'request_id': 'req-34e6fbee-a793-4fa8-a79a-af5e1d32f100'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:54.290741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee755f2-ad8f-429a-bfb6-ef4fafa8ba1f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:54.293951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-34e6fbee-a793-4fa8-a79a-af5e1d32f100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '221f816e91c04414b1fbb27dacc0bb41', 'request_id': 'req-34e6fbee-a793-4fa8-a79a-af5e1d32f100'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:54.303474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-34e6fbee-a793-4fa8-a79a-af5e1d32f100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '221f816e91c04414b1fbb27dacc0bb41', 'request_id': 'req-34e6fbee-a793-4fa8-a79a-af5e1d32f100'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:54.330456 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-34e6fbee-a793-4fa8-a79a-af5e1d32f100 admin admin] The token cache is being invalidate because project 221f816e91c04414b1fbb27dacc0bb41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:54.330798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-34e6fbee-a793-4fa8-a79a-af5e1d32f100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:54.331524 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-34e6fbee-a793-4fa8-a79a-af5e1d32f100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:54.334085 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1017/2028] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:54 2022] DELETE /identity/v3/projects/221f816e91c04414b1fbb27dacc0bb41 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:54.334625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6fbee-a793-4fa8-a79a-af5e1d32f100 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:54.336150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-34e6fbee-a793-4fa8-a79a-af5e1d32f100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:54.416839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487fb64e-ca0d-4519-8c80-05958c3fb223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:54.418265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fb64e-ca0d-4519-8c80-05958c3fb223 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.418547 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fb64e-ca0d-4519-8c80-05958c3fb223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.418814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487fb64e-ca0d-4519-8c80-05958c3fb223 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.420177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fb64e-ca0d-4519-8c80-05958c3fb223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:54.422343 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487fb64e-ca0d-4519-8c80-05958c3fb223 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:54.422741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6fbee-a793-4fa8-a79a-af5e1d32f100 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:54.431521 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04019d-eeab-4712-b2df-07f3334a003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:54.432329 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04019d-eeab-4712-b2df-07f3334a003a admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.432616 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04019d-eeab-4712-b2df-07f3334a003a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.432894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04019d-eeab-4712-b2df-07f3334a003a admin admin] PATH_INFO: `/v3/projects/993ec6132b1f4e8fbc4b16758f0532dd/users/8879ef61ea594ad39d77737750221cd4/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.454678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04019d-eeab-4712-b2df-07f3334a003a admin admin] RBAC: Authorizing `identity:create_grant(project_id=993ec6132b1f4e8fbc4b16758f0532dd, user_id=8879ef61ea594ad39d77737750221cd4, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:54.457571 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04019d-eeab-4712-b2df-07f3334a003a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:54.466749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1012/2029] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:54 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:54.466749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29182495-c596-4734-b76d-9fb23dfacc5a None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.466749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29182495-c596-4734-b76d-9fb23dfacc5a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.466749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29182495-c596-4734-b76d-9fb23dfacc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.479837 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1018/2030] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:54 2022] PUT /identity/v3/projects/993ec6132b1f4e8fbc4b16758f0532dd/users/8879ef61ea594ad39d77737750221cd4/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:54.488227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-29182495-c596-4734-b76d-9fb23dfacc5a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:54.489846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-29182495-c596-4734-b76d-9fb23dfacc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:54.489965 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04019d-eeab-4712-b2df-07f3334a003a admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:54.491775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9f04019d-eeab-4712-b2df-07f3334a003a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:54.570045 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1013/2031] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:54.576937 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade079e-aa77-4adb-a7f5-f2eafc8715ba None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.577496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade079e-aa77-4adb-a7f5-f2eafc8715ba None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.578648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade079e-aa77-4adb-a7f5-f2eafc8715ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.591684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04019d-eeab-4712-b2df-07f3334a003a admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:54.597147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d5ce2e-6854-46d6-aa00-72d5d74e2ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:54.597891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5ce2e-6854-46d6-aa00-72d5d74e2ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.598234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5ce2e-6854-46d6-aa00-72d5d74e2ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.598572 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d5ce2e-6854-46d6-aa00-72d5d74e2ee4 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.599750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5ce2e-6854-46d6-aa00-72d5d74e2ee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:54.601673 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d5ce2e-6854-46d6-aa00-72d5d74e2ee4 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:54.614814 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:54.625170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-bade079e-aa77-4adb-a7f5-f2eafc8715ba None None] MFA Rules not processed for user `8879ef61ea594ad39d77737750221cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:54.626635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-bade079e-aa77-4adb-a7f5-f2eafc8715ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:54.639081 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1019/2032] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:54 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:54.639081 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a51b3a-875b-4fa0-a2d1-f1ced55a6066 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.639081 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a51b3a-875b-4fa0-a2d1-f1ced55a6066 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.640383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a51b3a-875b-4fa0-a2d1-f1ced55a6066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.668674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-62a51b3a-875b-4fa0-a2d1-f1ced55a6066 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:54.669806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-62a51b3a-875b-4fa0-a2d1-f1ced55a6066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:54.684275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1020/2033] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:54.690392 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a51b3a-875b-4fa0-a2d1-f1ced55a6066 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:54.712002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a51b3a-875b-4fa0-a2d1-f1ced55a6066 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:54.725650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7e9f9-e0c6-4034-b613-2941fc87a032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:54.726293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7e9f9-e0c6-4034-b613-2941fc87a032 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.728843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7e9f9-e0c6-4034-b613-2941fc87a032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.728843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d7e9f9-e0c6-4034-b613-2941fc87a032 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.728843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7e9f9-e0c6-4034-b613-2941fc87a032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:54.729848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d7e9f9-e0c6-4034-b613-2941fc87a032 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:54.760476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1021/2034] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:54.762668 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7e9f9-e0c6-4034-b613-2941fc87a032 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:54.763701 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1014/2035] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:54.764853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade079e-aa77-4adb-a7f5-f2eafc8715ba None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:54.788893 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade079e-aa77-4adb-a7f5-f2eafc8715ba None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:54.791058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d7e9f9-e0c6-4034-b613-2941fc87a032 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:54.802044 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e2a34-3ca9-4c53-882d-f73bb0116226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:54.806413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e2a34-3ca9-4c53-882d-f73bb0116226 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.806718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e2a34-3ca9-4c53-882d-f73bb0116226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.807026 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e2a34-3ca9-4c53-882d-f73bb0116226 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.808564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e2a34-3ca9-4c53-882d-f73bb0116226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:08:54.808895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e2a34-3ca9-4c53-882d-f73bb0116226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:54.811805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e2a34-3ca9-4c53-882d-f73bb0116226 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:54.811936 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93baaa5b-8ae0-46d3-ad4c-b883a3d46555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:54.812450 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93baaa5b-8ae0-46d3-ad4c-b883a3d46555 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.812729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93baaa5b-8ae0-46d3-ad4c-b883a3d46555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.813427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93baaa5b-8ae0-46d3-ad4c-b883a3d46555 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.813642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93baaa5b-8ae0-46d3-ad4c-b883a3d46555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:54.815254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93baaa5b-8ae0-46d3-ad4c-b883a3d46555 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:54.826545 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:54.837410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1015/2036] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:54.847866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93baaa5b-8ae0-46d3-ad4c-b883a3d46555 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:54.849887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-93baaa5b-8ae0-46d3-ad4c-b883a3d46555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:54.875856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1022/2037] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:08:54.877396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e2a34-3ca9-4c53-882d-f73bb0116226 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:54.902046 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e2a34-3ca9-4c53-882d-f73bb0116226 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:54.917557 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90396c2b-68a0-4139-bfa4-5654b635a8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:54.918387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90396c2b-68a0-4139-bfa4-5654b635a8b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.918387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90396c2b-68a0-4139-bfa4-5654b635a8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.918626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90396c2b-68a0-4139-bfa4-5654b635a8b3 admin admin] PATH_INFO: `/v3/projects/763c3153ecd84b3fab6e77aef9c27802/users/440dfb858f6f4151967d1963b80d0b74/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.949528 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90396c2b-68a0-4139-bfa4-5654b635a8b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=763c3153ecd84b3fab6e77aef9c27802, user_id=440dfb858f6f4151967d1963b80d0b74, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:54.952249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90396c2b-68a0-4139-bfa4-5654b635a8b3 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:54.960988 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93baaa5b-8ae0-46d3-ad4c-b883a3d46555 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:54.970990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de7d9c-16d1-49c6-9e63-2f2421965fe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:54.972313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de7d9c-16d1-49c6-9e63-2f2421965fe3 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:54.973403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de7d9c-16d1-49c6-9e63-2f2421965fe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:54.973877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de7d9c-16d1-49c6-9e63-2f2421965fe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:54.979892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1023/2038] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:54 2022] PUT /identity/v3/projects/763c3153ecd84b3fab6e77aef9c27802/users/440dfb858f6f4151967d1963b80d0b74/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:54.987004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de7d9c-16d1-49c6-9e63-2f2421965fe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:54.989180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de7d9c-16d1-49c6-9e63-2f2421965fe3 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:54.991986 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90396c2b-68a0-4139-bfa4-5654b635a8b3 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:55.008252 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1016/2039] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:55.011558 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88d1f8-6594-41fd-b016-a8f1aedc2146 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:55.011951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88d1f8-6594-41fd-b016-a8f1aedc2146 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:55.014488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88d1f8-6594-41fd-b016-a8f1aedc2146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:55.081185 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90396c2b-68a0-4139-bfa4-5654b635a8b3 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:55.088618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3c8ad-9066-445c-939b-fb47de3d1258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:55.089284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3c8ad-9066-445c-939b-fb47de3d1258 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:55.089612 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3c8ad-9066-445c-939b-fb47de3d1258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:55.089892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3c8ad-9066-445c-939b-fb47de3d1258 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:55.090603 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3c8ad-9066-445c-939b-fb47de3d1258 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:55.092843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3c8ad-9066-445c-939b-fb47de3d1258 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:55.093630 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-2c88d1f8-6594-41fd-b016-a8f1aedc2146 None None] MFA Rules not processed for user `440dfb858f6f4151967d1963b80d0b74`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:55.098439 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2c88d1f8-6594-41fd-b016-a8f1aedc2146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:55.135731 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1024/2040] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:54 2022] POST /identity/v3/projects => generated 365 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:55.136900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3c8ad-9066-445c-939b-fb47de3d1258 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:55.270833 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1017/2041] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:55.272427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88d1f8-6594-41fd-b016-a8f1aedc2146 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:55.282396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3c8ad-9066-445c-939b-fb47de3d1258 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:55.287708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88d1f8-6594-41fd-b016-a8f1aedc2146 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:55.287864 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1a21b-7d54-4f30-a831-777be0997f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:55.288397 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1a21b-7d54-4f30-a831-777be0997f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:55.288686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1a21b-7d54-4f30-a831-777be0997f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:55.289103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1a21b-7d54-4f30-a831-777be0997f56 admin admin] PATH_INFO: `/v3/users/8879ef61ea594ad39d77737750221cd4` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:55.290559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1a21b-7d54-4f30-a831-777be0997f56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8879ef61ea594ad39d77737750221cd4)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:55.292674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1a21b-7d54-4f30-a831-777be0997f56 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:55.295285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2471d9-93b9-48aa-8e22-3c3edbbf4bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:55.296220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2471d9-93b9-48aa-8e22-3c3edbbf4bf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:55.296497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2471d9-93b9-48aa-8e22-3c3edbbf4bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:55.296766 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2471d9-93b9-48aa-8e22-3c3edbbf4bf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:55.297455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2471d9-93b9-48aa-8e22-3c3edbbf4bf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:55.299759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2471d9-93b9-48aa-8e22-3c3edbbf4bf7 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:55.321503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:INSERT=4,keystone:SELECT=92,keystone:DELETE=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:55.342848 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1018/2042] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:55 2022] POST /identity/v3/users => generated 283 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:55.343877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2471d9-93b9-48aa-8e22-3c3edbbf4bf7 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:55.359904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-55c1a21b-7d54-4f30-a831-777be0997f56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8879ef61ea594ad39d77737750221cd4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:55.372461 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-55c1a21b-7d54-4f30-a831-777be0997f56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8879ef61ea594ad39d77737750221cd4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:55.374371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-55c1a21b-7d54-4f30-a831-777be0997f56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8879ef61ea594ad39d77737750221cd4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:55.378881 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-55c1a21b-7d54-4f30-a831-777be0997f56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8879ef61ea594ad39d77737750221cd4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:55.379356 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2471d9-93b9-48aa-8e22-3c3edbbf4bf7 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:55.395150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d05497-572e-4a81-baa7-a85e5795050c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:55.395150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d05497-572e-4a81-baa7-a85e5795050c admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:55.396844 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d05497-572e-4a81-baa7-a85e5795050c admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:55.397150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d05497-572e-4a81-baa7-a85e5795050c admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:55.397792 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d05497-572e-4a81-baa7-a85e5795050c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:55.400138 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-55c1a21b-7d54-4f30-a831-777be0997f56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8879ef61ea594ad39d77737750221cd4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:55.404408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d05497-572e-4a81-baa7-a85e5795050c admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:55.425097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1025/2043] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:08:55 2022] DELETE /identity/v3/users/8879ef61ea594ad39d77737750221cd4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:55.426065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1a21b-7d54-4f30-a831-777be0997f56 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:55.428153 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-55c1a21b-7d54-4f30-a831-777be0997f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:55.447828 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1019/2044] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:55 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:55.449422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4be24e-2050-4680-a081-aae14a74dd9a None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:55.449755 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4be24e-2050-4680-a081-aae14a74dd9a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:55.450071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4be24e-2050-4680-a081-aae14a74dd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:55.486982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-ff4be24e-2050-4680-a081-aae14a74dd9a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:55.488781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ff4be24e-2050-4680-a081-aae14a74dd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:55.524573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1a21b-7d54-4f30-a831-777be0997f56 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:55.549807 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:55.570319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1020/2045] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:55.571807 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4be24e-2050-4680-a081-aae14a74dd9a None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:55.575987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ff4be24e-2050-4680-a081-aae14a74dd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:55.586695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4be24e-2050-4680-a081-aae14a74dd9a None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:55.592619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a782c6-79ac-431f-af86-84d73724a5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:55.593298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a782c6-79ac-431f-af86-84d73724a5bf admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:55.593580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a782c6-79ac-431f-af86-84d73724a5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:55.593812 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14366314-8ab3-4f25-bdfd-419657a106bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:55.593957 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a782c6-79ac-431f-af86-84d73724a5bf admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:55.594422 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14366314-8ab3-4f25-bdfd-419657a106bb admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:55.594623 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a782c6-79ac-431f-af86-84d73724a5bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:55.594732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14366314-8ab3-4f25-bdfd-419657a106bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:55.594948 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14366314-8ab3-4f25-bdfd-419657a106bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:55.595672 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14366314-8ab3-4f25-bdfd-419657a106bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:55.596617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a782c6-79ac-431f-af86-84d73724a5bf admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:55.597258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14366314-8ab3-4f25-bdfd-419657a106bb admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:55.607604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1026/2046] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:55.608337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14366314-8ab3-4f25-bdfd-419657a106bb admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:55.610494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-14366314-8ab3-4f25-bdfd-419657a106bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:55.629789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1021/2047] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:55 2022] POST /identity/v3/users => generated 288 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:55.629789 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a782c6-79ac-431f-af86-84d73724a5bf admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:55.657366 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a782c6-79ac-431f-af86-84d73724a5bf admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:55.666285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85862b98-f235-4909-8f07-206024d01349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:55.667190 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85862b98-f235-4909-8f07-206024d01349 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:55.667567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85862b98-f235-4909-8f07-206024d01349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:55.682269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85862b98-f235-4909-8f07-206024d01349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:55.682269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85862b98-f235-4909-8f07-206024d01349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:55.682269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85862b98-f235-4909-8f07-206024d01349 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:55.694964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1022/2048] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:55.694964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85862b98-f235-4909-8f07-206024d01349 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:55.708154 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85862b98-f235-4909-8f07-206024d01349 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:55.717735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:55.722958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb25441-e1e8-4ac3-8c11-408d43ec1002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:55.728683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb25441-e1e8-4ac3-8c11-408d43ec1002 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:55.729271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb25441-e1e8-4ac3-8c11-408d43ec1002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:55.729619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb25441-e1e8-4ac3-8c11-408d43ec1002 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:55.731312 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb25441-e1e8-4ac3-8c11-408d43ec1002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:55.739661 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14366314-8ab3-4f25-bdfd-419657a106bb admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:55.739926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb25441-e1e8-4ac3-8c11-408d43ec1002 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:55.744502 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134cf01-7e8b-442d-b770-d9cf9c44a159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:55.745220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134cf01-7e8b-442d-b770-d9cf9c44a159 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:55.745506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134cf01-7e8b-442d-b770-d9cf9c44a159 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:55.745773 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f134cf01-7e8b-442d-b770-d9cf9c44a159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:55.757017 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1023/2049] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:55.758015 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb25441-e1e8-4ac3-8c11-408d43ec1002 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:55.767651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134cf01-7e8b-442d-b770-d9cf9c44a159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:55.769096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f134cf01-7e8b-442d-b770-d9cf9c44a159 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:55.782578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb25441-e1e8-4ac3-8c11-408d43ec1002 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:55.788027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1027/2050] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:55.789145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134cf01-7e8b-442d-b770-d9cf9c44a159 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:55.793691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f7e74-3235-4259-af5f-112ed55e2f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:55.794408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f7e74-3235-4259-af5f-112ed55e2f82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:55.794691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f7e74-3235-4259-af5f-112ed55e2f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:55.794972 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f7e74-3235-4259-af5f-112ed55e2f82 admin admin] PATH_INFO: `/v3/projects/a38a6c663db24f2cadf9585f4a96ede5/users/0bbe0fb47d9f4a30bfd73c67eadefbc6/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:55.801903 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f134cf01-7e8b-442d-b770-d9cf9c44a159 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:55.809334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051296b-a048-494c-aa6d-d981f5441606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:55.810416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051296b-a048-494c-aa6d-d981f5441606 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:55.813573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051296b-a048-494c-aa6d-d981f5441606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:55.814180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e051296b-a048-494c-aa6d-d981f5441606 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:55.815102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051296b-a048-494c-aa6d-d981f5441606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:55.817383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e051296b-a048-494c-aa6d-d981f5441606 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:55.820100 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f7e74-3235-4259-af5f-112ed55e2f82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a38a6c663db24f2cadf9585f4a96ede5, user_id=0bbe0fb47d9f4a30bfd73c67eadefbc6, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:55.823506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f7e74-3235-4259-af5f-112ed55e2f82 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:55.832166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1028/2051] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:55.837607 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051296b-a048-494c-aa6d-d981f5441606 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:55.847723 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1024/2052] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:55 2022] PUT /identity/v3/projects/a38a6c663db24f2cadf9585f4a96ede5/users/0bbe0fb47d9f4a30bfd73c67eadefbc6/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:55.851985 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e051296b-a048-494c-aa6d-d981f5441606 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:55.855038 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f9d3f-eed0-4c0e-9642-3fe4903a4a0b None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:55.855683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f9d3f-eed0-4c0e-9642-3fe4903a4a0b None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:55.856017 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274f9d3f-eed0-4c0e-9642-3fe4903a4a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:55.917821 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-274f9d3f-eed0-4c0e-9642-3fe4903a4a0b None None] MFA Rules not processed for user `0bbe0fb47d9f4a30bfd73c67eadefbc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:55.920083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-274f9d3f-eed0-4c0e-9642-3fe4903a4a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:55.944800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b162de-aebf-4dc5-b97e-b44be8e6d670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:55.946488 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b162de-aebf-4dc5-b97e-b44be8e6d670 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:55.946976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b162de-aebf-4dc5-b97e-b44be8e6d670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:55.947520 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b162de-aebf-4dc5-b97e-b44be8e6d670 admin admin] PATH_INFO: `/v3/projects/38c18718c6ec41009c96dfad02b03f80/users/de337ddfc32447c5815d9412f1226871/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:55.965255 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b162de-aebf-4dc5-b97e-b44be8e6d670 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c18718c6ec41009c96dfad02b03f80, user_id=de337ddfc32447c5815d9412f1226871, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:55.972236 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b162de-aebf-4dc5-b97e-b44be8e6d670 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:56.001586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1029/2053] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:55 2022] PUT /identity/v3/projects/38c18718c6ec41009c96dfad02b03f80/users/de337ddfc32447c5815d9412f1226871/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:56.012269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37126e1-1058-4429-9abd-0d516e4958f6 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:56.012551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37126e1-1058-4429-9abd-0d516e4958f6 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:56.012847 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37126e1-1058-4429-9abd-0d516e4958f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:56.027485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1025/2054] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:56.032066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f9d3f-eed0-4c0e-9642-3fe4903a4a0b None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:56.047255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274f9d3f-eed0-4c0e-9642-3fe4903a4a0b None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:56.054391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-e37126e1-1058-4429-9abd-0d516e4958f6 None None] MFA Rules not processed for user `de337ddfc32447c5815d9412f1226871`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:56.056920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e37126e1-1058-4429-9abd-0d516e4958f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:56.149790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a69968-d82b-4961-acf0-c2aaed1bba57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:56.151018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a69968-d82b-4961-acf0-c2aaed1bba57 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:56.151354 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a69968-d82b-4961-acf0-c2aaed1bba57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:56.151663 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a69968-d82b-4961-acf0-c2aaed1bba57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:56.152465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a69968-d82b-4961-acf0-c2aaed1bba57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:56.154947 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a69968-d82b-4961-acf0-c2aaed1bba57 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:56.190378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1030/2055] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:08:56.195282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37126e1-1058-4429-9abd-0d516e4958f6 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:56.210240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37126e1-1058-4429-9abd-0d516e4958f6 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:56.216038 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1026/2056] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:56 2022] POST /identity/v3/projects => generated 367 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:08:56.219191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b79fb-d4ce-4593-bf3b-d8f70755ff21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:56.219753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a69968-d82b-4961-acf0-c2aaed1bba57 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:56.220238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b79fb-d4ce-4593-bf3b-d8f70755ff21 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:56.220894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b79fb-d4ce-4593-bf3b-d8f70755ff21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:56.220894 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b79fb-d4ce-4593-bf3b-d8f70755ff21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:56.221459 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b79fb-d4ce-4593-bf3b-d8f70755ff21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:56.227032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:56.228674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b79fb-d4ce-4593-bf3b-d8f70755ff21 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:56.305716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1031/2057] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:08:56 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:56.306744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b79fb-d4ce-4593-bf3b-d8f70755ff21 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:56.351814 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:56.358046 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a69968-d82b-4961-acf0-c2aaed1bba57 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:56.439402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b79fb-d4ce-4593-bf3b-d8f70755ff21 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:56.444031 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225933ae-5d33-43fd-a8a8-c582dcb6260a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:56.447431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225933ae-5d33-43fd-a8a8-c582dcb6260a admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:56.447960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225933ae-5d33-43fd-a8a8-c582dcb6260a admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:56.448323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225933ae-5d33-43fd-a8a8-c582dcb6260a admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:56.449300 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225933ae-5d33-43fd-a8a8-c582dcb6260a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:56.452415 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225933ae-5d33-43fd-a8a8-c582dcb6260a admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:56.454096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7633c-86af-47e5-8125-6a6de5a9acec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:56.454913 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7633c-86af-47e5-8125-6a6de5a9acec admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:56.455355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7633c-86af-47e5-8125-6a6de5a9acec admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:56.455820 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7633c-86af-47e5-8125-6a6de5a9acec admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:56.456625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7633c-86af-47e5-8125-6a6de5a9acec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:56.458959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7633c-86af-47e5-8125-6a6de5a9acec admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:56.484790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1027/2058] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:56 2022] POST /identity/v3/users => generated 290 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:56.488849 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225933ae-5d33-43fd-a8a8-c582dcb6260a admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:56.502212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1032/2059] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:08:56 2022] POST /identity/v3/users => generated 294 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:08:56.506694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7633c-86af-47e5-8125-6a6de5a9acec admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:56.507600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225933ae-5d33-43fd-a8a8-c582dcb6260a admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:56.519808 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7633c-86af-47e5-8125-6a6de5a9acec admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:56.523877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88071cc9-64d4-4ecf-ab51-f642e67308ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:56.524608 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88071cc9-64d4-4ecf-ab51-f642e67308ac admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:56.524960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88071cc9-64d4-4ecf-ab51-f642e67308ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:56.525306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88071cc9-64d4-4ecf-ab51-f642e67308ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:56.526106 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88071cc9-64d4-4ecf-ab51-f642e67308ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:56.527923 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88071cc9-64d4-4ecf-ab51-f642e67308ac admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:56.528393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a0e58-9e4f-4988-b204-25fdeeb8d6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:56.529249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a0e58-9e4f-4988-b204-25fdeeb8d6fc admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:56.530024 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a0e58-9e4f-4988-b204-25fdeeb8d6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:56.530570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a0e58-9e4f-4988-b204-25fdeeb8d6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:56.531836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a0e58-9e4f-4988-b204-25fdeeb8d6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:56.533694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843a0e58-9e4f-4988-b204-25fdeeb8d6fc admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:56.543740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1028/2060] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:08:56.546547 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88071cc9-64d4-4ecf-ab51-f642e67308ac admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:56.552508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1033/2061] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:08:56.557748 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a0e58-9e4f-4988-b204-25fdeeb8d6fc admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:56.561361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88071cc9-64d4-4ecf-ab51-f642e67308ac admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:56.570150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e2ba1-d6ac-40d1-aa34-2d55af78a505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:56.571563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e2ba1-d6ac-40d1-aa34-2d55af78a505 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:56.571681 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a0e58-9e4f-4988-b204-25fdeeb8d6fc admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:56.572146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e2ba1-d6ac-40d1-aa34-2d55af78a505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:56.572807 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e2ba1-d6ac-40d1-aa34-2d55af78a505 admin admin] PATH_INFO: `/v3/projects/8c4b42612bb447a9b5d8e7df7dbcda24/users/a31d9788b07a44c49009b35b7477b41b/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:56.581357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91cc696-8087-4a7a-a2d0-b139434fcee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:56.582137 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cc696-8087-4a7a-a2d0-b139434fcee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:56.582486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cc696-8087-4a7a-a2d0-b139434fcee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:56.582877 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91cc696-8087-4a7a-a2d0-b139434fcee0 admin admin] PATH_INFO: `/v3/projects/b2f4a79e58774ea9a36ae6b15f755e15/users/f9b8c5ee5332411993e6975940712e5f/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:56.597032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e2ba1-d6ac-40d1-aa34-2d55af78a505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c4b42612bb447a9b5d8e7df7dbcda24, user_id=a31d9788b07a44c49009b35b7477b41b, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:56.601291 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e2ba1-d6ac-40d1-aa34-2d55af78a505 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:56.611848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91cc696-8087-4a7a-a2d0-b139434fcee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2f4a79e58774ea9a36ae6b15f755e15, user_id=f9b8c5ee5332411993e6975940712e5f, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:56.615304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91cc696-8087-4a7a-a2d0-b139434fcee0 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:56.621751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1029/2062] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:56 2022] PUT /identity/v3/projects/8c4b42612bb447a9b5d8e7df7dbcda24/users/a31d9788b07a44c49009b35b7477b41b/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:56.627385 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e2ba1-d6ac-40d1-aa34-2d55af78a505 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:56.639504 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1034/2063] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:56 2022] PUT /identity/v3/projects/b2f4a79e58774ea9a36ae6b15f755e15/users/f9b8c5ee5332411993e6975940712e5f/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:08:56.644837 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91cc696-8087-4a7a-a2d0-b139434fcee0 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:56.751481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91cc696-8087-4a7a-a2d0-b139434fcee0 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:56.764067 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e6746-d84f-4892-8ee2-a66829b906e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:56.764709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e6746-d84f-4892-8ee2-a66829b906e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:56.764987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e6746-d84f-4892-8ee2-a66829b906e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:56.765256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e6746-d84f-4892-8ee2-a66829b906e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:56.765928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e6746-d84f-4892-8ee2-a66829b906e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:56.771630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44e6746-d84f-4892-8ee2-a66829b906e2 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:56.789892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1035/2064] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:56.793911 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e6746-d84f-4892-8ee2-a66829b906e2 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:56.794410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e2ba1-d6ac-40d1-aa34-2d55af78a505 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:56.800334 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a821688-295b-488f-9044-d609614e1aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:56.801220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a821688-295b-488f-9044-d609614e1aac admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:56.802447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a821688-295b-488f-9044-d609614e1aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:56.802867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a821688-295b-488f-9044-d609614e1aac admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:56.804289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a821688-295b-488f-9044-d609614e1aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:56.806077 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a821688-295b-488f-9044-d609614e1aac admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:56.806684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e6746-d84f-4892-8ee2-a66829b906e2 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:56.812768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf998d0c-1057-417b-b66b-a65ba4f351ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:56.813787 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf998d0c-1057-417b-b66b-a65ba4f351ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:56.814423 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf998d0c-1057-417b-b66b-a65ba4f351ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:56.815314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf998d0c-1057-417b-b66b-a65ba4f351ab admin admin] PATH_INFO: `/v3/projects/b2f4a79e58774ea9a36ae6b15f755e15/users/f9b8c5ee5332411993e6975940712e5f/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:56.817627 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf998d0c-1057-417b-b66b-a65ba4f351ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2f4a79e58774ea9a36ae6b15f755e15, user_id=f9b8c5ee5332411993e6975940712e5f, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:56.824294 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf998d0c-1057-417b-b66b-a65ba4f351ab admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:56.825958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1030/2065] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:08:56.827514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a821688-295b-488f-9044-d609614e1aac admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:56.836632 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:56.837868 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1036/2066] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:56 2022] PUT /identity/v3/projects/b2f4a79e58774ea9a36ae6b15f755e15/users/f9b8c5ee5332411993e6975940712e5f/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:56.843238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a821688-295b-488f-9044-d609614e1aac admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:56.849659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9bc58-f268-4aac-8b3d-9c93571d6f20 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:56.849932 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9bc58-f268-4aac-8b3d-9c93571d6f20 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:56.850207 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9bc58-f268-4aac-8b3d-9c93571d6f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:56.890182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-1aa9bc58-f268-4aac-8b3d-9c93571d6f20 None None] MFA Rules not processed for user `f9b8c5ee5332411993e6975940712e5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:56.892789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1aa9bc58-f268-4aac-8b3d-9c93571d6f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:56.930566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50a0e82-f6be-4dbe-900f-910d2eee6bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:56.932147 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a0e82-f6be-4dbe-900f-910d2eee6bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:56.932681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a0e82-f6be-4dbe-900f-910d2eee6bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:56.933345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50a0e82-f6be-4dbe-900f-910d2eee6bd8 admin admin] PATH_INFO: `/v3/projects/8c4b42612bb447a9b5d8e7df7dbcda24/users/a31d9788b07a44c49009b35b7477b41b/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:56.936676 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a0e82-f6be-4dbe-900f-910d2eee6bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c4b42612bb447a9b5d8e7df7dbcda24, user_id=a31d9788b07a44c49009b35b7477b41b, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:56.940290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50a0e82-f6be-4dbe-900f-910d2eee6bd8 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:56.955838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1031/2067] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:08:56 2022] PUT /identity/v3/projects/8c4b42612bb447a9b5d8e7df7dbcda24/users/a31d9788b07a44c49009b35b7477b41b/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:56.968953 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4da37-7500-4f32-a18d-0b2f67d50127 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:56.969655 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4da37-7500-4f32-a18d-0b2f67d50127 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:56.970185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4da37-7500-4f32-a18d-0b2f67d50127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:56.982642 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:57.009129 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-64f4da37-7500-4f32-a18d-0b2f67d50127 None None] MFA Rules not processed for user `a31d9788b07a44c49009b35b7477b41b`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:08:57.011523 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1037/2068] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:57.012110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-64f4da37-7500-4f32-a18d-0b2f67d50127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:57.025641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9bc58-f268-4aac-8b3d-9c93571d6f20 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:57.027646 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1aa9bc58-f268-4aac-8b3d-9c93571d6f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:57.148695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9bc58-f268-4aac-8b3d-9c93571d6f20 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:57.172693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1032/2069] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:08:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:08:57.180467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4da37-7500-4f32-a18d-0b2f67d50127 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:57.186773 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-64f4da37-7500-4f32-a18d-0b2f67d50127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:57.223358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760187f2-29bc-4835-83d6-1080dbb904af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:57.224744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760187f2-29bc-4835-83d6-1080dbb904af service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:57.225263 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760187f2-29bc-4835-83d6-1080dbb904af service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:57.225913 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760187f2-29bc-4835-83d6-1080dbb904af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:57.228878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-760187f2-29bc-4835-83d6-1080dbb904af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:57.324672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4da37-7500-4f32-a18d-0b2f67d50127 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:57.329423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485162b8-2381-4a2e-b81c-104d11954c5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:57.330800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485162b8-2381-4a2e-b81c-104d11954c5e service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:57.331076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485162b8-2381-4a2e-b81c-104d11954c5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:57.331345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485162b8-2381-4a2e-b81c-104d11954c5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:57.340075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760187f2-29bc-4835-83d6-1080dbb904af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:57.341496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760187f2-29bc-4835-83d6-1080dbb904af service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:57.344589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485162b8-2381-4a2e-b81c-104d11954c5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:57.346301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485162b8-2381-4a2e-b81c-104d11954c5e service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:57.359049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1033/2070] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:57 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:57.359942 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485162b8-2381-4a2e-b81c-104d11954c5e service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:57.361525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-485162b8-2381-4a2e-b81c-104d11954c5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:57.367182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1038/2071] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:08:57.409549 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:57.482508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485162b8-2381-4a2e-b81c-104d11954c5e service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:57.489031 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aefabd-b975-4c9f-b33a-526e690e5d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:57.489949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aefabd-b975-4c9f-b33a-526e690e5d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:57.490359 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aefabd-b975-4c9f-b33a-526e690e5d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:57.490801 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aefabd-b975-4c9f-b33a-526e690e5d70 admin admin] PATH_INFO: `/v3/projects/993ec6132b1f4e8fbc4b16758f0532dd` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:57.493418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aefabd-b975-4c9f-b33a-526e690e5d70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=993ec6132b1f4e8fbc4b16758f0532dd)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:57.501893 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aefabd-b975-4c9f-b33a-526e690e5d70 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:57.605233 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-04aefabd-b975-4c9f-b33a-526e690e5d70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '993ec6132b1f4e8fbc4b16758f0532dd', 'request_id': 'req-04aefabd-b975-4c9f-b33a-526e690e5d70'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:57.618093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-04aefabd-b975-4c9f-b33a-526e690e5d70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '993ec6132b1f4e8fbc4b16758f0532dd', 'request_id': 'req-04aefabd-b975-4c9f-b33a-526e690e5d70'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:57.636203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-04aefabd-b975-4c9f-b33a-526e690e5d70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '993ec6132b1f4e8fbc4b16758f0532dd', 'request_id': 'req-04aefabd-b975-4c9f-b33a-526e690e5d70'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:57.670629 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-04aefabd-b975-4c9f-b33a-526e690e5d70 admin admin] The token cache is being invalidate because project 993ec6132b1f4e8fbc4b16758f0532dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:08:57.670954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-04aefabd-b975-4c9f-b33a-526e690e5d70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:57.672954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-04aefabd-b975-4c9f-b33a-526e690e5d70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:08:57.678034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1034/2072] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:08:57 2022] DELETE /identity/v3/projects/993ec6132b1f4e8fbc4b16758f0532dd => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:08:57.751329 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760187f2-29bc-4835-83d6-1080dbb904af service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:57.762099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760187f2-29bc-4835-83d6-1080dbb904af service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:57.962161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98523d0-bd5b-4fc7-bbca-3d76268bda18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:57.963318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98523d0-bd5b-4fc7-bbca-3d76268bda18 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:57.964850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98523d0-bd5b-4fc7-bbca-3d76268bda18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:57.966449 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98523d0-bd5b-4fc7-bbca-3d76268bda18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:57.969854 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f98523d0-bd5b-4fc7-bbca-3d76268bda18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:58.000559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:08:58.102541 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98523d0-bd5b-4fc7-bbca-3d76268bda18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:58.112132 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98523d0-bd5b-4fc7-bbca-3d76268bda18 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:58.161871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1039/2073] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:08:59.324542 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aefabd-b975-4c9f-b33a-526e690e5d70 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:08:59.326048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-04aefabd-b975-4c9f-b33a-526e690e5d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:59.341975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aefabd-b975-4c9f-b33a-526e690e5d70 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:08:59.346412 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d2089-777e-4ac6-9996-0c83f39eefa2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:08:59.347232 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d2089-777e-4ac6-9996-0c83f39eefa2 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:08:59.347682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d2089-777e-4ac6-9996-0c83f39eefa2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:08:59.348090 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d2089-777e-4ac6-9996-0c83f39eefa2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:08:59.349975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e11d2089-777e-4ac6-9996-0c83f39eefa2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:08:59.417284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d2089-777e-4ac6-9996-0c83f39eefa2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:08:59.418901 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d2089-777e-4ac6-9996-0c83f39eefa2 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:08:59.429457 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1035/2074] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:08:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:09:00.344976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98523d0-bd5b-4fc7-bbca-3d76268bda18 service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:00.355387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98523d0-bd5b-4fc7-bbca-3d76268bda18 service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:00.363788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d39ab7f-c8ed-4a23-86d5-6fbbbc429323 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:00.363788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39ab7f-c8ed-4a23-86d5-6fbbbc429323 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:00.363788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39ab7f-c8ed-4a23-86d5-6fbbbc429323 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:00.363788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39ab7f-c8ed-4a23-86d5-6fbbbc429323 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:00.365069 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0d39ab7f-c8ed-4a23-86d5-6fbbbc429323 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:00.435469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39ab7f-c8ed-4a23-86d5-6fbbbc429323 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:00.437011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d39ab7f-c8ed-4a23-86d5-6fbbbc429323 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:00.447666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1040/2075] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:09:01.968307 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d2089-777e-4ac6-9996-0c83f39eefa2 service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:01.969716 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e11d2089-777e-4ac6-9996-0c83f39eefa2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:02.023125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:02.038634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d2089-777e-4ac6-9996-0c83f39eefa2 service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:02.043914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442101e-db39-4b47-86ec-d36ad3350620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:02.044555 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442101e-db39-4b47-86ec-d36ad3350620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:02.044831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442101e-db39-4b47-86ec-d36ad3350620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:02.045100 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442101e-db39-4b47-86ec-d36ad3350620 admin admin] PATH_INFO: `/v3/users/0bbe0fb47d9f4a30bfd73c67eadefbc6` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:02.046237 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442101e-db39-4b47-86ec-d36ad3350620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bbe0fb47d9f4a30bfd73c67eadefbc6)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:02.049192 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442101e-db39-4b47-86ec-d36ad3350620 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:02.104491 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-7442101e-db39-4b47-86ec-d36ad3350620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bbe0fb47d9f4a30bfd73c67eadefbc6'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:02.112486 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-7442101e-db39-4b47-86ec-d36ad3350620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bbe0fb47d9f4a30bfd73c67eadefbc6'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:02.113120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-7442101e-db39-4b47-86ec-d36ad3350620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bbe0fb47d9f4a30bfd73c67eadefbc6'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:02.113758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-7442101e-db39-4b47-86ec-d36ad3350620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bbe0fb47d9f4a30bfd73c67eadefbc6'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:02.126440 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-7442101e-db39-4b47-86ec-d36ad3350620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bbe0fb47d9f4a30bfd73c67eadefbc6'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:02.139146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1036/2076] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:01 2022] DELETE /identity/v3/users/0bbe0fb47d9f4a30bfd73c67eadefbc6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:02.143863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05d830-cfba-40f0-8ec5-bc574a93592a None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:02.144368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05d830-cfba-40f0-8ec5-bc574a93592a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:02.144470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05d830-cfba-40f0-8ec5-bc574a93592a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:02.167590 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-ca05d830-cfba-40f0-8ec5-bc574a93592a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:02.168832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ca05d830-cfba-40f0-8ec5-bc574a93592a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:02.229060 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1041/2077] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:02.240180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442101e-db39-4b47-86ec-d36ad3350620 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:02.241849 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-7442101e-db39-4b47-86ec-d36ad3350620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:02.302513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442101e-db39-4b47-86ec-d36ad3350620 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:02.306416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25ae72-b2fe-45ea-83be-486985b43e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:02.307174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25ae72-b2fe-45ea-83be-486985b43e40 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:02.307585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25ae72-b2fe-45ea-83be-486985b43e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:02.307932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c25ae72-b2fe-45ea-83be-486985b43e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:02.317969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25ae72-b2fe-45ea-83be-486985b43e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:02.319564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c25ae72-b2fe-45ea-83be-486985b43e40 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:02.329743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1037/2078] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:09:02.610792 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05d830-cfba-40f0-8ec5-bc574a93592a None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:02.612308 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ca05d830-cfba-40f0-8ec5-bc574a93592a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:02.624263 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05d830-cfba-40f0-8ec5-bc574a93592a None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:02.629027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af711d4e-d924-43f7-8910-5339fabbe916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:02.630192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af711d4e-d924-43f7-8910-5339fabbe916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:02.630192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af711d4e-d924-43f7-8910-5339fabbe916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:02.630192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af711d4e-d924-43f7-8910-5339fabbe916 admin admin] PATH_INFO: `/v3/projects/a38a6c663db24f2cadf9585f4a96ede5` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:02.631330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af711d4e-d924-43f7-8910-5339fabbe916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a38a6c663db24f2cadf9585f4a96ede5)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:02.633258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af711d4e-d924-43f7-8910-5339fabbe916 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:02.678821 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:02.683208 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-af711d4e-d924-43f7-8910-5339fabbe916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a38a6c663db24f2cadf9585f4a96ede5', 'request_id': 'req-af711d4e-d924-43f7-8910-5339fabbe916'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:02.692932 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-af711d4e-d924-43f7-8910-5339fabbe916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a38a6c663db24f2cadf9585f4a96ede5', 'request_id': 'req-af711d4e-d924-43f7-8910-5339fabbe916'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:02.699740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-af711d4e-d924-43f7-8910-5339fabbe916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a38a6c663db24f2cadf9585f4a96ede5', 'request_id': 'req-af711d4e-d924-43f7-8910-5339fabbe916'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:02.711683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-af711d4e-d924-43f7-8910-5339fabbe916 admin admin] The token cache is being invalidate because project a38a6c663db24f2cadf9585f4a96ede5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:02.711973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-af711d4e-d924-43f7-8910-5339fabbe916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:02.712636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-af711d4e-d924-43f7-8910-5339fabbe916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:02.714206 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1042/2079] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:02 2022] DELETE /identity/v3/projects/a38a6c663db24f2cadf9585f4a96ede5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:02.718425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25ae72-b2fe-45ea-83be-486985b43e40 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:02.728499 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c25ae72-b2fe-45ea-83be-486985b43e40 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:02.795013 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b86f0-b33f-4064-be7c-056c61e36615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:02.795692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b86f0-b33f-4064-be7c-056c61e36615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:02.795973 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b86f0-b33f-4064-be7c-056c61e36615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:02.796251 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b86f0-b33f-4064-be7c-056c61e36615 admin admin] PATH_INFO: `/v3/users/a31d9788b07a44c49009b35b7477b41b` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:02.797434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b86f0-b33f-4064-be7c-056c61e36615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a31d9788b07a44c49009b35b7477b41b)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:02.799389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b86f0-b33f-4064-be7c-056c61e36615 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:02.815588 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:02.849948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ae9b86f0-b33f-4064-be7c-056c61e36615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a31d9788b07a44c49009b35b7477b41b'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:02.858223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ae9b86f0-b33f-4064-be7c-056c61e36615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a31d9788b07a44c49009b35b7477b41b'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:02.858760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ae9b86f0-b33f-4064-be7c-056c61e36615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a31d9788b07a44c49009b35b7477b41b'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:02.859369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ae9b86f0-b33f-4064-be7c-056c61e36615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a31d9788b07a44c49009b35b7477b41b'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:02.871407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ae9b86f0-b33f-4064-be7c-056c61e36615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a31d9788b07a44c49009b35b7477b41b'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:02.884612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1038/2080] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:02 2022] DELETE /identity/v3/users/a31d9788b07a44c49009b35b7477b41b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:03.299797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af711d4e-d924-43f7-8910-5339fabbe916 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:03.301811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-af711d4e-d924-43f7-8910-5339fabbe916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:03.338842 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af711d4e-d924-43f7-8910-5339fabbe916 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:03.423633 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47e9e0-e3bc-40b7-ad3b-ff28bfd5b089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:03.424273 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47e9e0-e3bc-40b7-ad3b-ff28bfd5b089 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:03.424545 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47e9e0-e3bc-40b7-ad3b-ff28bfd5b089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:03.424814 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47e9e0-e3bc-40b7-ad3b-ff28bfd5b089 admin admin] PATH_INFO: `/v3/projects/8c4b42612bb447a9b5d8e7df7dbcda24` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:03.425922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47e9e0-e3bc-40b7-ad3b-ff28bfd5b089 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c4b42612bb447a9b5d8e7df7dbcda24)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:03.427842 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47e9e0-e3bc-40b7-ad3b-ff28bfd5b089 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:03.477928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-7f47e9e0-e3bc-40b7-ad3b-ff28bfd5b089 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c4b42612bb447a9b5d8e7df7dbcda24', 'request_id': 'req-7f47e9e0-e3bc-40b7-ad3b-ff28bfd5b089'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:03.487371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-7f47e9e0-e3bc-40b7-ad3b-ff28bfd5b089 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c4b42612bb447a9b5d8e7df7dbcda24', 'request_id': 'req-7f47e9e0-e3bc-40b7-ad3b-ff28bfd5b089'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:03.493302 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-7f47e9e0-e3bc-40b7-ad3b-ff28bfd5b089 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c4b42612bb447a9b5d8e7df7dbcda24', 'request_id': 'req-7f47e9e0-e3bc-40b7-ad3b-ff28bfd5b089'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:03.504818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-7f47e9e0-e3bc-40b7-ad3b-ff28bfd5b089 admin admin] The token cache is being invalidate because project 8c4b42612bb447a9b5d8e7df7dbcda24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:03.505179 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-7f47e9e0-e3bc-40b7-ad3b-ff28bfd5b089 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:03.505779 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-7f47e9e0-e3bc-40b7-ad3b-ff28bfd5b089 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:03.507382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1043/2081] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:03 2022] DELETE /identity/v3/projects/8c4b42612bb447a9b5d8e7df7dbcda24 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:03.512493 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a41695-86ce-4fcb-bdd5-909ccfe0de96 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:03.512964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a41695-86ce-4fcb-bdd5-909ccfe0de96 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:03.513063 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a41695-86ce-4fcb-bdd5-909ccfe0de96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:03.534502 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-23a41695-86ce-4fcb-bdd5-909ccfe0de96 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:03.535836 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-23a41695-86ce-4fcb-bdd5-909ccfe0de96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:03.600086 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1039/2082] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:03.608389 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18a683-5da3-41f4-9f08-e3a2eee0f08c None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:03.608689 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18a683-5da3-41f4-9f08-e3a2eee0f08c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:03.608926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18a683-5da3-41f4-9f08-e3a2eee0f08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:03.628462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-7a18a683-5da3-41f4-9f08-e3a2eee0f08c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:03.629633 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-7a18a683-5da3-41f4-9f08-e3a2eee0f08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:03.637495 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1044/2083] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:03.641787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a41695-86ce-4fcb-bdd5-909ccfe0de96 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:03.652006 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a41695-86ce-4fcb-bdd5-909ccfe0de96 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:03.656723 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316097a-6429-48e6-8d94-0db02e7e28a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:03.657336 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316097a-6429-48e6-8d94-0db02e7e28a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:03.657608 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316097a-6429-48e6-8d94-0db02e7e28a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:03.657879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316097a-6429-48e6-8d94-0db02e7e28a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:03.658527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316097a-6429-48e6-8d94-0db02e7e28a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:09:03.658842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316097a-6429-48e6-8d94-0db02e7e28a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:03.660534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316097a-6429-48e6-8d94-0db02e7e28a5 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:03.672581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1040/2084] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:09:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:09:03.677386 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18a683-5da3-41f4-9f08-e3a2eee0f08c None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:03.686847 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18a683-5da3-41f4-9f08-e3a2eee0f08c None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:03.692420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92535d2a-979b-407a-8a36-7d801c1fa832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:03.693031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535d2a-979b-407a-8a36-7d801c1fa832 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:03.693298 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535d2a-979b-407a-8a36-7d801c1fa832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:03.693558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92535d2a-979b-407a-8a36-7d801c1fa832 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:03.694173 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92535d2a-979b-407a-8a36-7d801c1fa832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:03.696476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92535d2a-979b-407a-8a36-7d801c1fa832 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:03.727019 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1045/2085] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:09:03 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:03.731423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316097a-6429-48e6-8d94-0db02e7e28a5 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:03.741879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316097a-6429-48e6-8d94-0db02e7e28a5 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:03.798364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef487d3-366c-421e-9f26-5c29216a50eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:03.798948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef487d3-366c-421e-9f26-5c29216a50eb admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:03.799222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef487d3-366c-421e-9f26-5c29216a50eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:03.799483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef487d3-366c-421e-9f26-5c29216a50eb admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:03.800140 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef487d3-366c-421e-9f26-5c29216a50eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:03.802094 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef487d3-366c-421e-9f26-5c29216a50eb admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:03.824286 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1041/2086] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:09:03 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:03.829133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92535d2a-979b-407a-8a36-7d801c1fa832 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:03.842472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92535d2a-979b-407a-8a36-7d801c1fa832 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:03.848282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8480e-cab3-42bd-9c10-7ba26c7b0b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:03.848883 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8480e-cab3-42bd-9c10-7ba26c7b0b1f admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:03.849157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8480e-cab3-42bd-9c10-7ba26c7b0b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:03.849417 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8480e-cab3-42bd-9c10-7ba26c7b0b1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:03.850048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8480e-cab3-42bd-9c10-7ba26c7b0b1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:03.851698 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8480e-cab3-42bd-9c10-7ba26c7b0b1f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:03.862102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1046/2087] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:03.866009 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef487d3-366c-421e-9f26-5c29216a50eb admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:03.876579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef487d3-366c-421e-9f26-5c29216a50eb admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:03.881606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ae221-1839-4ab7-8ba0-bb09bdcb3e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:03.882147 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ae221-1839-4ab7-8ba0-bb09bdcb3e63 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:03.882425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ae221-1839-4ab7-8ba0-bb09bdcb3e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:03.882700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ae221-1839-4ab7-8ba0-bb09bdcb3e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:03.883345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ae221-1839-4ab7-8ba0-bb09bdcb3e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:03.885021 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ae221-1839-4ab7-8ba0-bb09bdcb3e63 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:03.890236 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:03.899999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1042/2088] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:03.901441 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8480e-cab3-42bd-9c10-7ba26c7b0b1f admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:03.913777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8480e-cab3-42bd-9c10-7ba26c7b0b1f admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:03.918945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c793f5c-4e0a-4d8f-affa-a764e4d77c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:03.919651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c793f5c-4e0a-4d8f-affa-a764e4d77c3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:03.919934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c793f5c-4e0a-4d8f-affa-a764e4d77c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:03.920211 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c793f5c-4e0a-4d8f-affa-a764e4d77c3f admin admin] PATH_INFO: `/v3/projects/dc3d52bd127049faba00bc0e59a7c979/users/d51a22c9048e4c53a29abb58522ae361/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:03.935436 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c793f5c-4e0a-4d8f-affa-a764e4d77c3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc3d52bd127049faba00bc0e59a7c979, user_id=d51a22c9048e4c53a29abb58522ae361, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:03.938180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c793f5c-4e0a-4d8f-affa-a764e4d77c3f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:03.948438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:03.954780 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1047/2089] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:03 2022] PUT /identity/v3/projects/dc3d52bd127049faba00bc0e59a7c979/users/d51a22c9048e4c53a29abb58522ae361/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:03.965606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d88ee-5c1f-4a91-92a8-43c2cdc5525c None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:03.965920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d88ee-5c1f-4a91-92a8-43c2cdc5525c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:03.966191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d88ee-5c1f-4a91-92a8-43c2cdc5525c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:03.994093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-4d2d88ee-5c1f-4a91-92a8-43c2cdc5525c None None] MFA Rules not processed for user `d51a22c9048e4c53a29abb58522ae361`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:03.995359 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-4d2d88ee-5c1f-4a91-92a8-43c2cdc5525c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:04.059588 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1043/2090] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:04.064261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c793f5c-4e0a-4d8f-affa-a764e4d77c3f admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:04.127187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c793f5c-4e0a-4d8f-affa-a764e4d77c3f admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:04.132769 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0f757-6959-4f62-acf5-6a9b009e025a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:04.133311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0f757-6959-4f62-acf5-6a9b009e025a admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:04.133595 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0f757-6959-4f62-acf5-6a9b009e025a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:04.133868 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0f757-6959-4f62-acf5-6a9b009e025a admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:04.134509 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0f757-6959-4f62-acf5-6a9b009e025a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:04.136473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0f757-6959-4f62-acf5-6a9b009e025a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:04.168589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1048/2091] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:09:04 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:04.172573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d88ee-5c1f-4a91-92a8-43c2cdc5525c None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:04.183487 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d88ee-5c1f-4a91-92a8-43c2cdc5525c None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:04.244954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf9104-2b5c-4e5c-ba98-51d603978960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:04.245586 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf9104-2b5c-4e5c-ba98-51d603978960 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:04.245861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf9104-2b5c-4e5c-ba98-51d603978960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:04.246124 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf9104-2b5c-4e5c-ba98-51d603978960 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:04.246760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf9104-2b5c-4e5c-ba98-51d603978960 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:04.248718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf9104-2b5c-4e5c-ba98-51d603978960 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:04.278476 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1044/2092] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:09:04 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:09:04.282454 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0f757-6959-4f62-acf5-6a9b009e025a admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:04.293970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0f757-6959-4f62-acf5-6a9b009e025a admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:04.304331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0042dd2-b468-4f35-99f6-89ac5f95868a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:04.304977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0042dd2-b468-4f35-99f6-89ac5f95868a admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:04.305226 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0042dd2-b468-4f35-99f6-89ac5f95868a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:04.305496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0042dd2-b468-4f35-99f6-89ac5f95868a admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:04.306169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0042dd2-b468-4f35-99f6-89ac5f95868a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:04.311327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0042dd2-b468-4f35-99f6-89ac5f95868a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:04.331433 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1049/2093] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:04.335692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf9104-2b5c-4e5c-ba98-51d603978960 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:04.345740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf9104-2b5c-4e5c-ba98-51d603978960 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:04.350583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55845d-1176-4242-b247-34d1b6a322cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:04.351215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55845d-1176-4242-b247-34d1b6a322cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:04.351488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55845d-1176-4242-b247-34d1b6a322cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:04.351777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b55845d-1176-4242-b247-34d1b6a322cb admin admin] PATH_INFO: `/v3/projects/6f78b9fb0d13415083c245dd5cafc57a/users/ba18d473553a4e21b97c5be486842373/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:04.366930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55845d-1176-4242-b247-34d1b6a322cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f78b9fb0d13415083c245dd5cafc57a, user_id=ba18d473553a4e21b97c5be486842373, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:04.369670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b55845d-1176-4242-b247-34d1b6a322cb admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:04.394647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1045/2094] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:04 2022] PUT /identity/v3/projects/6f78b9fb0d13415083c245dd5cafc57a/users/ba18d473553a4e21b97c5be486842373/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:04.399416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0042dd2-b468-4f35-99f6-89ac5f95868a admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:04.412113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0042dd2-b468-4f35-99f6-89ac5f95868a admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:04.448877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55845d-1176-4242-b247-34d1b6a322cb admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:04.451036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8b55845d-1176-4242-b247-34d1b6a322cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:04.499684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdba1f7-3f4d-4e88-bab5-b9452561f964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:04.500482 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdba1f7-3f4d-4e88-bab5-b9452561f964 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:04.500758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdba1f7-3f4d-4e88-bab5-b9452561f964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:04.501101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdba1f7-3f4d-4e88-bab5-b9452561f964 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:04.501885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdba1f7-3f4d-4e88-bab5-b9452561f964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:04.503775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdba1f7-3f4d-4e88-bab5-b9452561f964 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:04.521835 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1050/2095] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:04.526324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdba1f7-3f4d-4e88-bab5-b9452561f964 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:04.530440 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:04.536824 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b55845d-1176-4242-b247-34d1b6a322cb admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:04.540533 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdba1f7-3f4d-4e88-bab5-b9452561f964 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:04.543671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3991b-5a34-4700-a1f3-ac7699152a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:04.544416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3991b-5a34-4700-a1f3-ac7699152a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:04.544873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3991b-5a34-4700-a1f3-ac7699152a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:04.545238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3991b-5a34-4700-a1f3-ac7699152a5f admin admin] PATH_INFO: `/v3/users/440dfb858f6f4151967d1963b80d0b74` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:04.546471 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3991b-5a34-4700-a1f3-ac7699152a5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=440dfb858f6f4151967d1963b80d0b74)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:04.547173 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf4e04b-56ad-4ee2-9bee-d06a98066e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:04.547900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4e04b-56ad-4ee2-9bee-d06a98066e3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:04.548650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4e04b-56ad-4ee2-9bee-d06a98066e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:04.548650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4e04b-56ad-4ee2-9bee-d06a98066e3f admin admin] PATH_INFO: `/v3/projects/6f78b9fb0d13415083c245dd5cafc57a/users/ba18d473553a4e21b97c5be486842373/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:04.548842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3991b-5a34-4700-a1f3-ac7699152a5f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:04.550671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4e04b-56ad-4ee2-9bee-d06a98066e3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f78b9fb0d13415083c245dd5cafc57a, user_id=ba18d473553a4e21b97c5be486842373, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:04.555119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf4e04b-56ad-4ee2-9bee-d06a98066e3f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:04.565822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1051/2096] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:04 2022] PUT /identity/v3/projects/6f78b9fb0d13415083c245dd5cafc57a/users/ba18d473553a4e21b97c5be486842373/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:09:04.578048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389e704-51e9-44f9-9539-7fa9865d115c None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:04.578333 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389e704-51e9-44f9-9539-7fa9865d115c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:04.578611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389e704-51e9-44f9-9539-7fa9865d115c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:04.626931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-d389e704-51e9-44f9-9539-7fa9865d115c None None] MFA Rules not processed for user `ba18d473553a4e21b97c5be486842373`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:04.628570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d389e704-51e9-44f9-9539-7fa9865d115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:04.628706 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-92f3991b-5a34-4700-a1f3-ac7699152a5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '440dfb858f6f4151967d1963b80d0b74'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:04.640879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-92f3991b-5a34-4700-a1f3-ac7699152a5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '440dfb858f6f4151967d1963b80d0b74'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:04.642781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-92f3991b-5a34-4700-a1f3-ac7699152a5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '440dfb858f6f4151967d1963b80d0b74'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:04.643796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-92f3991b-5a34-4700-a1f3-ac7699152a5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '440dfb858f6f4151967d1963b80d0b74'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:04.664228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-92f3991b-5a34-4700-a1f3-ac7699152a5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '440dfb858f6f4151967d1963b80d0b74'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:04.683530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1046/2097] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:04 2022] DELETE /identity/v3/users/440dfb858f6f4151967d1963b80d0b74 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:09:04.688511 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6805d-fa2e-458c-99c8-1d652176a440 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:04.689009 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6805d-fa2e-458c-99c8-1d652176a440 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:04.689411 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c6805d-fa2e-458c-99c8-1d652176a440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:04.693705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:04.733245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-55c6805d-fa2e-458c-99c8-1d652176a440 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:04.734872 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-55c6805d-fa2e-458c-99c8-1d652176a440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:04.741745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1052/2098] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:09:04.754488 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389e704-51e9-44f9-9539-7fa9865d115c None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:04.756087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d389e704-51e9-44f9-9539-7fa9865d115c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:04.808453 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1047/2099] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:04.818132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6805d-fa2e-458c-99c8-1d652176a440 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:04.819438 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-55c6805d-fa2e-458c-99c8-1d652176a440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:04.819571 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389e704-51e9-44f9-9539-7fa9865d115c None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:04.868673 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de3801-9d38-43ca-849d-941b55e4e076 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:04.869284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de3801-9d38-43ca-849d-941b55e4e076 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:04.869615 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de3801-9d38-43ca-849d-941b55e4e076 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:04.869880 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de3801-9d38-43ca-849d-941b55e4e076 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:04.876220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c6805d-fa2e-458c-99c8-1d652176a440 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:04.879855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a09b1-eee7-4c5a-b737-23fe281dbde8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:04.880465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a09b1-eee7-4c5a-b737-23fe281dbde8 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:04.880740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a09b1-eee7-4c5a-b737-23fe281dbde8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:04.881009 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a09b1-eee7-4c5a-b737-23fe281dbde8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:04.890423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a09b1-eee7-4c5a-b737-23fe281dbde8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:04.892024 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a09b1-eee7-4c5a-b737-23fe281dbde8 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:04.901402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1048/2100] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:09:04.929056 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de3801-9d38-43ca-849d-941b55e4e076 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:04.930499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de3801-9d38-43ca-849d-941b55e4e076 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:04.940925 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1053/2101] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:09:05.084589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a09b1-eee7-4c5a-b737-23fe281dbde8 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:05.086390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-cb8a09b1-eee7-4c5a-b737-23fe281dbde8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:05.099807 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a09b1-eee7-4c5a-b737-23fe281dbde8 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:05.105050 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a828deaf-276d-4dce-851c-de7658dc571b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:05.105672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828deaf-276d-4dce-851c-de7658dc571b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:05.105948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828deaf-276d-4dce-851c-de7658dc571b admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:05.106239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828deaf-276d-4dce-851c-de7658dc571b admin admin] PATH_INFO: `/v3/users/d51a22c9048e4c53a29abb58522ae361` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:05.107510 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828deaf-276d-4dce-851c-de7658dc571b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d51a22c9048e4c53a29abb58522ae361)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:05.109447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a828deaf-276d-4dce-851c-de7658dc571b admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:05.122034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:05.161192 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a828deaf-276d-4dce-851c-de7658dc571b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd51a22c9048e4c53a29abb58522ae361'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.170220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a828deaf-276d-4dce-851c-de7658dc571b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd51a22c9048e4c53a29abb58522ae361'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.171276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a828deaf-276d-4dce-851c-de7658dc571b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd51a22c9048e4c53a29abb58522ae361'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.172203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a828deaf-276d-4dce-851c-de7658dc571b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd51a22c9048e4c53a29abb58522ae361'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.184745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de3801-9d38-43ca-849d-941b55e4e076 service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:05.184745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-11de3801-9d38-43ca-849d-941b55e4e076 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:05.192538 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a828deaf-276d-4dce-851c-de7658dc571b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd51a22c9048e4c53a29abb58522ae361'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.196153 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de3801-9d38-43ca-849d-941b55e4e076 service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:05.201907 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaeee2e-0c38-4e50-aa80-492d361a13fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:05.202633 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaeee2e-0c38-4e50-aa80-492d361a13fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:05.202981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaeee2e-0c38-4e50-aa80-492d361a13fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:05.203313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaeee2e-0c38-4e50-aa80-492d361a13fc admin admin] PATH_INFO: `/v3/projects/763c3153ecd84b3fab6e77aef9c27802` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:05.204573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaeee2e-0c38-4e50-aa80-492d361a13fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=763c3153ecd84b3fab6e77aef9c27802)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:05.206596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaeee2e-0c38-4e50-aa80-492d361a13fc admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:05.207783 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1049/2102] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:05 2022] DELETE /identity/v3/users/d51a22c9048e4c53a29abb58522ae361 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:05.212973 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b32022-a230-4c75-b5b4-fdd1afd77c06 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:05.213287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b32022-a230-4c75-b5b4-fdd1afd77c06 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:05.213620 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b32022-a230-4c75-b5b4-fdd1afd77c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:05.233381 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-c8b32022-a230-4c75-b5b4-fdd1afd77c06 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:05.234585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c8b32022-a230-4c75-b5b4-fdd1afd77c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:05.253045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:05.263120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6eaeee2e-0c38-4e50-aa80-492d361a13fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '763c3153ecd84b3fab6e77aef9c27802', 'request_id': 'req-6eaeee2e-0c38-4e50-aa80-492d361a13fc'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.271722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6eaeee2e-0c38-4e50-aa80-492d361a13fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '763c3153ecd84b3fab6e77aef9c27802', 'request_id': 'req-6eaeee2e-0c38-4e50-aa80-492d361a13fc'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.278827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6eaeee2e-0c38-4e50-aa80-492d361a13fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '763c3153ecd84b3fab6e77aef9c27802', 'request_id': 'req-6eaeee2e-0c38-4e50-aa80-492d361a13fc'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.290464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6eaeee2e-0c38-4e50-aa80-492d361a13fc admin admin] The token cache is being invalidate because project 763c3153ecd84b3fab6e77aef9c27802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:05.290866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6eaeee2e-0c38-4e50-aa80-492d361a13fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.291580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6eaeee2e-0c38-4e50-aa80-492d361a13fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.293103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1054/2103] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:05 2022] DELETE /identity/v3/projects/763c3153ecd84b3fab6e77aef9c27802 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:09:05.294898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1050/2104] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:09:05.298022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaeee2e-0c38-4e50-aa80-492d361a13fc admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:05.298202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6eaeee2e-0c38-4e50-aa80-492d361a13fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:05.304446 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b32022-a230-4c75-b5b4-fdd1afd77c06 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:05.305983 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c8b32022-a230-4c75-b5b4-fdd1afd77c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:05.362588 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b32022-a230-4c75-b5b4-fdd1afd77c06 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:05.363855 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaeee2e-0c38-4e50-aa80-492d361a13fc admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:05.368917 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805f38f-1a65-4288-8d9a-0af0b56152b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:05.369621 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805f38f-1a65-4288-8d9a-0af0b56152b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:05.369903 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805f38f-1a65-4288-8d9a-0af0b56152b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:05.370330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805f38f-1a65-4288-8d9a-0af0b56152b9 admin admin] PATH_INFO: `/v3/users/de337ddfc32447c5815d9412f1226871` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:05.372023 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805f38f-1a65-4288-8d9a-0af0b56152b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de337ddfc32447c5815d9412f1226871)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:05.373960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805f38f-1a65-4288-8d9a-0af0b56152b9 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:05.407955 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:05.414315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af950af4-0c86-4c68-9b15-a6a97e56c97e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:05.414887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af950af4-0c86-4c68-9b15-a6a97e56c97e service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:05.415165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af950af4-0c86-4c68-9b15-a6a97e56c97e service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:05.415471 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af950af4-0c86-4c68-9b15-a6a97e56c97e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:05.418259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-af950af4-0c86-4c68-9b15-a6a97e56c97e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:05.423197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1805f38f-1a65-4288-8d9a-0af0b56152b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de337ddfc32447c5815d9412f1226871'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.431224 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1805f38f-1a65-4288-8d9a-0af0b56152b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de337ddfc32447c5815d9412f1226871'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.432070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1805f38f-1a65-4288-8d9a-0af0b56152b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de337ddfc32447c5815d9412f1226871'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.432771 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1805f38f-1a65-4288-8d9a-0af0b56152b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de337ddfc32447c5815d9412f1226871'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.433964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af950af4-0c86-4c68-9b15-a6a97e56c97e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:05.435415 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af950af4-0c86-4c68-9b15-a6a97e56c97e service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:05.439937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1805f38f-1a65-4288-8d9a-0af0b56152b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de337ddfc32447c5815d9412f1226871'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.445717 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1051/2105] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:09:05.457960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1055/2106] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:05 2022] DELETE /identity/v3/users/de337ddfc32447c5815d9412f1226871 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:05.724253 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af950af4-0c86-4c68-9b15-a6a97e56c97e service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:05.725820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-af950af4-0c86-4c68-9b15-a6a97e56c97e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:05.744155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af950af4-0c86-4c68-9b15-a6a97e56c97e service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:05.823977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c510b-e072-49c4-b954-62f3f3c2a0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:05.824743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c510b-e072-49c4-b954-62f3f3c2a0cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:05.825120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c510b-e072-49c4-b954-62f3f3c2a0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:05.825546 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c510b-e072-49c4-b954-62f3f3c2a0cb admin admin] PATH_INFO: `/v3/projects/dc3d52bd127049faba00bc0e59a7c979` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:05.827765 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c510b-e072-49c4-b954-62f3f3c2a0cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc3d52bd127049faba00bc0e59a7c979)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:05.832711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c510b-e072-49c4-b954-62f3f3c2a0cb admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:05.834122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805f38f-1a65-4288-8d9a-0af0b56152b9 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:05.836121 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1805f38f-1a65-4288-8d9a-0af0b56152b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:05.873439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805f38f-1a65-4288-8d9a-0af0b56152b9 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:05.878811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ff03e-9f89-4d04-b503-50a815496575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:05.880417 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ff03e-9f89-4d04-b503-50a815496575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:05.880912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ff03e-9f89-4d04-b503-50a815496575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:05.881881 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ff03e-9f89-4d04-b503-50a815496575 admin admin] PATH_INFO: `/v3/projects/38c18718c6ec41009c96dfad02b03f80` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:05.883613 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ff03e-9f89-4d04-b503-50a815496575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38c18718c6ec41009c96dfad02b03f80)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:05.886268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ff03e-9f89-4d04-b503-50a815496575 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:05.931136 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-876c510b-e072-49c4-b954-62f3f3c2a0cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc3d52bd127049faba00bc0e59a7c979', 'request_id': 'req-876c510b-e072-49c4-b954-62f3f3c2a0cb'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.942065 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-876c510b-e072-49c4-b954-62f3f3c2a0cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc3d52bd127049faba00bc0e59a7c979', 'request_id': 'req-876c510b-e072-49c4-b954-62f3f3c2a0cb'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.949190 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-876c510b-e072-49c4-b954-62f3f3c2a0cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc3d52bd127049faba00bc0e59a7c979', 'request_id': 'req-876c510b-e072-49c4-b954-62f3f3c2a0cb'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.952026 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:05.958485 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-964ff03e-9f89-4d04-b503-50a815496575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38c18718c6ec41009c96dfad02b03f80', 'request_id': 'req-964ff03e-9f89-4d04-b503-50a815496575'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.961996 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-876c510b-e072-49c4-b954-62f3f3c2a0cb admin admin] The token cache is being invalidate because project dc3d52bd127049faba00bc0e59a7c979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:05.962296 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-876c510b-e072-49c4-b954-62f3f3c2a0cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.962956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-876c510b-e072-49c4-b954-62f3f3c2a0cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.964678 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1052/2107] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:05 2022] DELETE /identity/v3/projects/dc3d52bd127049faba00bc0e59a7c979 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:05.968446 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-964ff03e-9f89-4d04-b503-50a815496575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38c18718c6ec41009c96dfad02b03f80', 'request_id': 'req-964ff03e-9f89-4d04-b503-50a815496575'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.968824 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c510b-e072-49c4-b954-62f3f3c2a0cb admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:05.970234 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-876c510b-e072-49c4-b954-62f3f3c2a0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:05.975491 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-964ff03e-9f89-4d04-b503-50a815496575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38c18718c6ec41009c96dfad02b03f80', 'request_id': 'req-964ff03e-9f89-4d04-b503-50a815496575'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.988236 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-964ff03e-9f89-4d04-b503-50a815496575 admin admin] The token cache is being invalidate because project 38c18718c6ec41009c96dfad02b03f80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:05.988534 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-964ff03e-9f89-4d04-b503-50a815496575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.989526 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-964ff03e-9f89-4d04-b503-50a815496575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:05.991797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1056/2108] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:05 2022] DELETE /identity/v3/projects/38c18718c6ec41009c96dfad02b03f80 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:05.995331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ff03e-9f89-4d04-b503-50a815496575 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:05.996802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-964ff03e-9f89-4d04-b503-50a815496575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:06.041600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c510b-e072-49c4-b954-62f3f3c2a0cb admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:06.065611 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:06.076846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ff03e-9f89-4d04-b503-50a815496575 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:06.081765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033efb0a-17ee-456b-9b92-c9e0983fd6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:06.082476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033efb0a-17ee-456b-9b92-c9e0983fd6b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:06.082855 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033efb0a-17ee-456b-9b92-c9e0983fd6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:06.083130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033efb0a-17ee-456b-9b92-c9e0983fd6b9 admin admin] PATH_INFO: `/v3/users/f9b8c5ee5332411993e6975940712e5f` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:06.084321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033efb0a-17ee-456b-9b92-c9e0983fd6b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9b8c5ee5332411993e6975940712e5f)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:06.086256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033efb0a-17ee-456b-9b92-c9e0983fd6b9 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:06.099397 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7ffcc-7242-43fc-b2ee-1857b57d1957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:06.100071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7ffcc-7242-43fc-b2ee-1857b57d1957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:06.100332 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7ffcc-7242-43fc-b2ee-1857b57d1957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:06.100597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c7ffcc-7242-43fc-b2ee-1857b57d1957 admin admin] PATH_INFO: `/v3/users/ba18d473553a4e21b97c5be486842373` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:06.101721 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7ffcc-7242-43fc-b2ee-1857b57d1957 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba18d473553a4e21b97c5be486842373)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:06.103794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c7ffcc-7242-43fc-b2ee-1857b57d1957 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:06.135479 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-033efb0a-17ee-456b-9b92-c9e0983fd6b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9b8c5ee5332411993e6975940712e5f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.143151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-033efb0a-17ee-456b-9b92-c9e0983fd6b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9b8c5ee5332411993e6975940712e5f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.143901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-033efb0a-17ee-456b-9b92-c9e0983fd6b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9b8c5ee5332411993e6975940712e5f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.144677 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-033efb0a-17ee-456b-9b92-c9e0983fd6b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9b8c5ee5332411993e6975940712e5f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.151689 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-033efb0a-17ee-456b-9b92-c9e0983fd6b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9b8c5ee5332411993e6975940712e5f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.153117 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d3c7ffcc-7242-43fc-b2ee-1857b57d1957 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba18d473553a4e21b97c5be486842373'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.161472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d3c7ffcc-7242-43fc-b2ee-1857b57d1957 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba18d473553a4e21b97c5be486842373'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.162116 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d3c7ffcc-7242-43fc-b2ee-1857b57d1957 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba18d473553a4e21b97c5be486842373'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.162914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d3c7ffcc-7242-43fc-b2ee-1857b57d1957 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba18d473553a4e21b97c5be486842373'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.170635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1057/2109] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:05 2022] DELETE /identity/v3/users/f9b8c5ee5332411993e6975940712e5f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:09:06.184747 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d3c7ffcc-7242-43fc-b2ee-1857b57d1957 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba18d473553a4e21b97c5be486842373'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.198085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1053/2110] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:05 2022] DELETE /identity/v3/users/ba18d473553a4e21b97c5be486842373 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:06.437410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033efb0a-17ee-456b-9b92-c9e0983fd6b9 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:06.438938 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-033efb0a-17ee-456b-9b92-c9e0983fd6b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:06.485985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7ffcc-7242-43fc-b2ee-1857b57d1957 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:06.487355 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d3c7ffcc-7242-43fc-b2ee-1857b57d1957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:06.506232 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:06.525396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033efb0a-17ee-456b-9b92-c9e0983fd6b9 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:06.558527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c7ffcc-7242-43fc-b2ee-1857b57d1957 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:06.563492 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910d31d-56cc-42e7-9294-f38b25a8d044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:06.564398 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910d31d-56cc-42e7-9294-f38b25a8d044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:06.564995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910d31d-56cc-42e7-9294-f38b25a8d044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:06.565467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910d31d-56cc-42e7-9294-f38b25a8d044 admin admin] PATH_INFO: `/v3/projects/6f78b9fb0d13415083c245dd5cafc57a` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:06.566930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910d31d-56cc-42e7-9294-f38b25a8d044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f78b9fb0d13415083c245dd5cafc57a)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:06.569099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910d31d-56cc-42e7-9294-f38b25a8d044 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:06.591175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc13d910-44c6-4009-acc8-001b5c8098d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:06.591848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13d910-44c6-4009-acc8-001b5c8098d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:06.592130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13d910-44c6-4009-acc8-001b5c8098d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:06.592402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc13d910-44c6-4009-acc8-001b5c8098d1 admin admin] PATH_INFO: `/v3/projects/b2f4a79e58774ea9a36ae6b15f755e15` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:06.593582 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13d910-44c6-4009-acc8-001b5c8098d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2f4a79e58774ea9a36ae6b15f755e15)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:06.595560 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc13d910-44c6-4009-acc8-001b5c8098d1 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:06.645988 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:06.647955 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d910d31d-56cc-42e7-9294-f38b25a8d044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f78b9fb0d13415083c245dd5cafc57a', 'request_id': 'req-d910d31d-56cc-42e7-9294-f38b25a8d044'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.662110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d910d31d-56cc-42e7-9294-f38b25a8d044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f78b9fb0d13415083c245dd5cafc57a', 'request_id': 'req-d910d31d-56cc-42e7-9294-f38b25a8d044'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.667420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-fc13d910-44c6-4009-acc8-001b5c8098d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2f4a79e58774ea9a36ae6b15f755e15', 'request_id': 'req-fc13d910-44c6-4009-acc8-001b5c8098d1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.672315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d910d31d-56cc-42e7-9294-f38b25a8d044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f78b9fb0d13415083c245dd5cafc57a', 'request_id': 'req-d910d31d-56cc-42e7-9294-f38b25a8d044'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.677233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-fc13d910-44c6-4009-acc8-001b5c8098d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2f4a79e58774ea9a36ae6b15f755e15', 'request_id': 'req-fc13d910-44c6-4009-acc8-001b5c8098d1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.685818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-fc13d910-44c6-4009-acc8-001b5c8098d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2f4a79e58774ea9a36ae6b15f755e15', 'request_id': 'req-fc13d910-44c6-4009-acc8-001b5c8098d1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.688976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d910d31d-56cc-42e7-9294-f38b25a8d044 admin admin] The token cache is being invalidate because project 6f78b9fb0d13415083c245dd5cafc57a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:06.689359 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d910d31d-56cc-42e7-9294-f38b25a8d044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.690091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d910d31d-56cc-42e7-9294-f38b25a8d044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.692286 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1054/2111] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:06 2022] DELETE /identity/v3/projects/6f78b9fb0d13415083c245dd5cafc57a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:09:06.697311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435adc5-8d0c-4478-ab6a-06149e3e54aa None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:06.697658 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435adc5-8d0c-4478-ab6a-06149e3e54aa None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:06.698061 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435adc5-8d0c-4478-ab6a-06149e3e54aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:06.698335 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-fc13d910-44c6-4009-acc8-001b5c8098d1 admin admin] The token cache is being invalidate because project b2f4a79e58774ea9a36ae6b15f755e15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:06.698631 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-fc13d910-44c6-4009-acc8-001b5c8098d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.699270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-fc13d910-44c6-4009-acc8-001b5c8098d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:06.704025 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1058/2112] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:06 2022] DELETE /identity/v3/projects/b2f4a79e58774ea9a36ae6b15f755e15 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:06.705972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08c8c8-705b-41f3-9a8d-47d93ef7efd1 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:06.706254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08c8c8-705b-41f3-9a8d-47d93ef7efd1 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:06.706521 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08c8c8-705b-41f3-9a8d-47d93ef7efd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:06.719204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-4435adc5-8d0c-4478-ab6a-06149e3e54aa None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:06.720443 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-4435adc5-8d0c-4478-ab6a-06149e3e54aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:06.726733 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-0e08c8c8-705b-41f3-9a8d-47d93ef7efd1 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:06.727953 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0e08c8c8-705b-41f3-9a8d-47d93ef7efd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:06.783131 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1055/2113] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:09:06.789705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1059/2114] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:06.791278 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c167633-c502-4935-9bf4-627035747956 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:06.791587 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c167633-c502-4935-9bf4-627035747956 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:06.791859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c167633-c502-4935-9bf4-627035747956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:06.797112 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa7cd2-34a2-46ac-8fac-09c5c4d4d161 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:06.797397 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa7cd2-34a2-46ac-8fac-09c5c4d4d161 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:06.797672 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa7cd2-34a2-46ac-8fac-09c5c4d4d161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:06.811247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-5c167633-c502-4935-9bf4-627035747956 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:06.812462 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5c167633-c502-4935-9bf4-627035747956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:06.816600 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-0baa7cd2-34a2-46ac-8fac-09c5c4d4d161 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:06.817781 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0baa7cd2-34a2-46ac-8fac-09c5c4d4d161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:06.820518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1056/2115] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:06.824722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c167633-c502-4935-9bf4-627035747956 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:06.826962 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1060/2116] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:09:06.831018 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa7cd2-34a2-46ac-8fac-09c5c4d4d161 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:06.834009 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c167633-c502-4935-9bf4-627035747956 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:06.838647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb086e-d437-4573-b6fa-f0fc89d7a238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:06.839230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb086e-d437-4573-b6fa-f0fc89d7a238 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:06.839537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb086e-d437-4573-b6fa-f0fc89d7a238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:06.839801 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febb086e-d437-4573-b6fa-f0fc89d7a238 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:06.840061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa7cd2-34a2-46ac-8fac-09c5c4d4d161 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:06.840437 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb086e-d437-4573-b6fa-f0fc89d7a238 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:09:06.840732 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb086e-d437-4573-b6fa-f0fc89d7a238 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:06.842310 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febb086e-d437-4573-b6fa-f0fc89d7a238 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:06.844514 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ec2b5-2bc3-4d62-828c-92351f073503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:06.845101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ec2b5-2bc3-4d62-828c-92351f073503 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:06.845377 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ec2b5-2bc3-4d62-828c-92351f073503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:06.845646 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473ec2b5-2bc3-4d62-828c-92351f073503 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:06.846314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ec2b5-2bc3-4d62-828c-92351f073503 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:09:06.846614 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ec2b5-2bc3-4d62-828c-92351f073503 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:06.848237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473ec2b5-2bc3-4d62-828c-92351f073503 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:06.854231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1057/2117] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:09:06.858145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb086e-d437-4573-b6fa-f0fc89d7a238 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:06.863201 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1061/2118] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:09:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:09:06.864267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ec2b5-2bc3-4d62-828c-92351f073503 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:06.869957 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febb086e-d437-4573-b6fa-f0fc89d7a238 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:06.875469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473ec2b5-2bc3-4d62-828c-92351f073503 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:06.875611 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfef3fd-7156-4a28-9f10-cf1ca491361a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:06.876261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfef3fd-7156-4a28-9f10-cf1ca491361a admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:06.876403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfef3fd-7156-4a28-9f10-cf1ca491361a admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:06.876749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfef3fd-7156-4a28-9f10-cf1ca491361a admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:06.877290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfef3fd-7156-4a28-9f10-cf1ca491361a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:06.879244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfef3fd-7156-4a28-9f10-cf1ca491361a admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:06.880847 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98802ba-07c5-48e5-a16f-7a5abea19b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:06.881576 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98802ba-07c5-48e5-a16f-7a5abea19b18 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:06.881856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98802ba-07c5-48e5-a16f-7a5abea19b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:06.882123 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98802ba-07c5-48e5-a16f-7a5abea19b18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:06.882857 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98802ba-07c5-48e5-a16f-7a5abea19b18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:06.884802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98802ba-07c5-48e5-a16f-7a5abea19b18 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:06.890459 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:06.914270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1058/2119] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:09:06 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:09:06.915141 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfef3fd-7156-4a28-9f10-cf1ca491361a admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:06.930410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1062/2120] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:09:06 2022] POST /identity/v3/projects => generated 365 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:06.931368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98802ba-07c5-48e5-a16f-7a5abea19b18 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:07.002953 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98802ba-07c5-48e5-a16f-7a5abea19b18 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:07.008095 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be622e-4400-43b5-b046-cbf65c7ee723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:07.009079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be622e-4400-43b5-b046-cbf65c7ee723 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.009360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be622e-4400-43b5-b046-cbf65c7ee723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.010311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be622e-4400-43b5-b046-cbf65c7ee723 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.010978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be622e-4400-43b5-b046-cbf65c7ee723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:07.013211 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be622e-4400-43b5-b046-cbf65c7ee723 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:07.014445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfef3fd-7156-4a28-9f10-cf1ca491361a admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:07.020330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fd681-0d23-45db-8bea-b4ab7bc67fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:07.021045 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fd681-0d23-45db-8bea-b4ab7bc67fb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.021542 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fd681-0d23-45db-8bea-b4ab7bc67fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.021873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fd681-0d23-45db-8bea-b4ab7bc67fb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.022582 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fd681-0d23-45db-8bea-b4ab7bc67fb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:07.025207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fd681-0d23-45db-8bea-b4ab7bc67fb2 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:07.045086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1063/2121] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:09:06 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:09:07.046305 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be622e-4400-43b5-b046-cbf65c7ee723 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:07.056575 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be622e-4400-43b5-b046-cbf65c7ee723 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:07.061654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b2cef-de22-4919-827f-c203deb3bab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:07.062654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b2cef-de22-4919-827f-c203deb3bab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.062918 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b2cef-de22-4919-827f-c203deb3bab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.063244 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b2cef-de22-4919-827f-c203deb3bab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.064144 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b2cef-de22-4919-827f-c203deb3bab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:07.065864 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b2cef-de22-4919-827f-c203deb3bab5 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:07.068889 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1059/2122] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:09:06 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:07.072808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fd681-0d23-45db-8bea-b4ab7bc67fb2 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:07.080330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1064/2123] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:07.086036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b2cef-de22-4919-827f-c203deb3bab5 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:07.089081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fd681-0d23-45db-8bea-b4ab7bc67fb2 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:07.094713 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dacde-3b66-4ed9-909e-2cb4f192569e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:07.095690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dacde-3b66-4ed9-909e-2cb4f192569e admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.096023 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dacde-3b66-4ed9-909e-2cb4f192569e admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.096298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dacde-3b66-4ed9-909e-2cb4f192569e admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.097263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65dacde-3b66-4ed9-909e-2cb4f192569e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:07.098471 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b2cef-de22-4919-827f-c203deb3bab5 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:07.099536 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65dacde-3b66-4ed9-909e-2cb4f192569e admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:07.105566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fef264-8506-4f16-9667-359b04869332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:07.106503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fef264-8506-4f16-9667-359b04869332 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.107396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fef264-8506-4f16-9667-359b04869332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.108149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fef264-8506-4f16-9667-359b04869332 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.109319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fef264-8506-4f16-9667-359b04869332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:07.111164 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fef264-8506-4f16-9667-359b04869332 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:07.116407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1060/2124] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:07.118813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dacde-3b66-4ed9-909e-2cb4f192569e admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:07.130994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1065/2125] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:07.134998 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fef264-8506-4f16-9667-359b04869332 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:07.144182 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dacde-3b66-4ed9-909e-2cb4f192569e admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:07.149010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fef264-8506-4f16-9667-359b04869332 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:07.150068 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb104e-d388-4af5-8ad0-51672679a7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:07.151142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb104e-d388-4af5-8ad0-51672679a7bb admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.151422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb104e-d388-4af5-8ad0-51672679a7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.151724 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb104e-d388-4af5-8ad0-51672679a7bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.152377 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb104e-d388-4af5-8ad0-51672679a7bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:07.153997 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb104e-d388-4af5-8ad0-51672679a7bb admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:07.154516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674e320-eed6-41c3-b859-bf878591af7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:07.155169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674e320-eed6-41c3-b859-bf878591af7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.155697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674e320-eed6-41c3-b859-bf878591af7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.156076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674e320-eed6-41c3-b859-bf878591af7b admin admin] PATH_INFO: `/v3/projects/4f49764f81904d7faa8a203eef70ec24/users/2776b06e421d4d388c4d987596770f76/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.170516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1061/2126] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:07.172498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb104e-d388-4af5-8ad0-51672679a7bb admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:07.183498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674e320-eed6-41c3-b859-bf878591af7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f49764f81904d7faa8a203eef70ec24, user_id=2776b06e421d4d388c4d987596770f76, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:07.184107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:07.184773 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674e320-eed6-41c3-b859-bf878591af7b admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:07.191258 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb104e-d388-4af5-8ad0-51672679a7bb admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:07.200549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1066/2127] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:07 2022] PUT /identity/v3/projects/4f49764f81904d7faa8a203eef70ec24/users/2776b06e421d4d388c4d987596770f76/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:09:07.201213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdfd1c9-8955-491c-a3b1-cc658f8a93f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:07.202092 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfd1c9-8955-491c-a3b1-cc658f8a93f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.202563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfd1c9-8955-491c-a3b1-cc658f8a93f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.202886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdfd1c9-8955-491c-a3b1-cc658f8a93f9 admin admin] PATH_INFO: `/v3/projects/fdeafe2f60fe4bc68663e8f93437038e/users/d4ea7c5a840b4048bb649812d07fc5cb/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.211156 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3040d6-daff-42ef-a004-1dea12f0e83c None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.211809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3040d6-daff-42ef-a004-1dea12f0e83c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.211809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3040d6-daff-42ef-a004-1dea12f0e83c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.219476 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfd1c9-8955-491c-a3b1-cc658f8a93f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdeafe2f60fe4bc68663e8f93437038e, user_id=d4ea7c5a840b4048bb649812d07fc5cb, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:07.222306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdfd1c9-8955-491c-a3b1-cc658f8a93f9 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:07.238149 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1062/2128] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:07 2022] PUT /identity/v3/projects/fdeafe2f60fe4bc68663e8f93437038e/users/d4ea7c5a840b4048bb649812d07fc5cb/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:07.249851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab7570-6318-496e-8b6b-6a816915bc6d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.250074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab7570-6318-496e-8b6b-6a816915bc6d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.250343 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab7570-6318-496e-8b6b-6a816915bc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.263807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-5b3040d6-daff-42ef-a004-1dea12f0e83c None None] MFA Rules not processed for user `2776b06e421d4d388c4d987596770f76`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:07.265088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5b3040d6-daff-42ef-a004-1dea12f0e83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:07.279755 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-55ab7570-6318-496e-8b6b-6a816915bc6d None None] MFA Rules not processed for user `d4ea7c5a840b4048bb649812d07fc5cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:07.281648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-55ab7570-6318-496e-8b6b-6a816915bc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:07.352827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1067/2129] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:09:07.365308 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3040d6-daff-42ef-a004-1dea12f0e83c None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:07.366882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5b3040d6-daff-42ef-a004-1dea12f0e83c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:07.367649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1063/2130] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:09:07.371880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab7570-6318-496e-8b6b-6a816915bc6d None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:07.391905 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:07.433867 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3040d6-daff-42ef-a004-1dea12f0e83c None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:07.438712 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d6655-6cc8-4178-a3b8-08787af2562c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:07.439394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d6655-6cc8-4178-a3b8-08787af2562c service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.439800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d6655-6cc8-4178-a3b8-08787af2562c service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.440112 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d6655-6cc8-4178-a3b8-08787af2562c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.440688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab7570-6318-496e-8b6b-6a816915bc6d None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:07.445734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd8ce8-7b3d-4455-aa0a-a527345f3acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:07.446347 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd8ce8-7b3d-4455-aa0a-a527345f3acf admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.446621 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd8ce8-7b3d-4455-aa0a-a527345f3acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.446885 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdd8ce8-7b3d-4455-aa0a-a527345f3acf admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.448489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd8ce8-7b3d-4455-aa0a-a527345f3acf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:07.449536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d6655-6cc8-4178-a3b8-08787af2562c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:07.450503 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdd8ce8-7b3d-4455-aa0a-a527345f3acf admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:07.450968 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2d6655-6cc8-4178-a3b8-08787af2562c service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:07.460689 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1068/2131] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:09:07.493502 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1064/2132] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:09:07 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:07.495411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d6655-6cc8-4178-a3b8-08787af2562c service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:07.507464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2d6655-6cc8-4178-a3b8-08787af2562c service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:07.579323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708c568-0374-4302-b979-82709e77da48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:07.580164 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708c568-0374-4302-b979-82709e77da48 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.580644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708c568-0374-4302-b979-82709e77da48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.581073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708c568-0374-4302-b979-82709e77da48 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.581827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708c568-0374-4302-b979-82709e77da48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:07.584078 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708c568-0374-4302-b979-82709e77da48 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:07.614520 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1069/2133] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:09:07 2022] POST /identity/v3/users => generated 290 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:07.618963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd8ce8-7b3d-4455-aa0a-a527345f3acf admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:07.630380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdd8ce8-7b3d-4455-aa0a-a527345f3acf admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:07.638286 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a06c31-7d2a-4901-9e36-e7e9698ae8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:07.639016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a06c31-7d2a-4901-9e36-e7e9698ae8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.639299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a06c31-7d2a-4901-9e36-e7e9698ae8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.639616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a06c31-7d2a-4901-9e36-e7e9698ae8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.640373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a06c31-7d2a-4901-9e36-e7e9698ae8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:07.642185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a06c31-7d2a-4901-9e36-e7e9698ae8a8 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:07.655480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1065/2134] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:07.660880 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708c568-0374-4302-b979-82709e77da48 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:07.677416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708c568-0374-4302-b979-82709e77da48 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:07.683331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403401e-d0d2-45d7-a5cb-9149e008f41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:07.684081 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403401e-d0d2-45d7-a5cb-9149e008f41e admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.684357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403401e-d0d2-45d7-a5cb-9149e008f41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.684641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403401e-d0d2-45d7-a5cb-9149e008f41e admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.685393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403401e-d0d2-45d7-a5cb-9149e008f41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:07.687282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2403401e-d0d2-45d7-a5cb-9149e008f41e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:07.704569 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1070/2135] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:07.704720 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a06c31-7d2a-4901-9e36-e7e9698ae8a8 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:07.719675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a06c31-7d2a-4901-9e36-e7e9698ae8a8 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:07.736799 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3602fb-6d9d-4062-a72d-ea01508b896d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:07.737640 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3602fb-6d9d-4062-a72d-ea01508b896d admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.738039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3602fb-6d9d-4062-a72d-ea01508b896d admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.738212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3602fb-6d9d-4062-a72d-ea01508b896d admin admin] PATH_INFO: `/v3/projects/98081f3b934c4e6383ff7abb0f071975/users/2a5b0e85206a4dfaaa4b4634ad1c7a1d/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.768750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3602fb-6d9d-4062-a72d-ea01508b896d admin admin] RBAC: Authorizing `identity:create_grant(project_id=98081f3b934c4e6383ff7abb0f071975, user_id=2a5b0e85206a4dfaaa4b4634ad1c7a1d, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:07.772042 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3602fb-6d9d-4062-a72d-ea01508b896d admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:07.806350 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1066/2136] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:07 2022] PUT /identity/v3/projects/98081f3b934c4e6383ff7abb0f071975/users/2a5b0e85206a4dfaaa4b4634ad1c7a1d/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:07.824075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0186d-4df8-43f3-9c7c-3d993cb86ad6 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:07.824354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0186d-4df8-43f3-9c7c-3d993cb86ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:07.824644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0186d-4df8-43f3-9c7c-3d993cb86ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:07.869015 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-47a0186d-4df8-43f3-9c7c-3d993cb86ad6 None None] MFA Rules not processed for user `2a5b0e85206a4dfaaa4b4634ad1c7a1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:07.870430 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-47a0186d-4df8-43f3-9c7c-3d993cb86ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:07.956777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1071/2137] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:07.961640 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3602fb-6d9d-4062-a72d-ea01508b896d admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:07.983399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:08.032635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3602fb-6d9d-4062-a72d-ea01508b896d admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:08.037376 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d70fc-d3d9-4e4c-8393-01163fa7a34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:08.038103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d70fc-d3d9-4e4c-8393-01163fa7a34a admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:08.038441 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d70fc-d3d9-4e4c-8393-01163fa7a34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:08.038716 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d70fc-d3d9-4e4c-8393-01163fa7a34a admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:08.039389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d70fc-d3d9-4e4c-8393-01163fa7a34a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:08.041428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d70fc-d3d9-4e4c-8393-01163fa7a34a admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:08.077738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1067/2138] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:09:07 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:09:08.081788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0186d-4df8-43f3-9c7c-3d993cb86ad6 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:08.099883 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0186d-4df8-43f3-9c7c-3d993cb86ad6 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:08.140327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:08.188489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4880ed-e2fa-42f5-8981-d94d7b30a4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:08.189194 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4880ed-e2fa-42f5-8981-d94d7b30a4a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:08.189565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4880ed-e2fa-42f5-8981-d94d7b30a4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:08.189900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4880ed-e2fa-42f5-8981-d94d7b30a4a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:08.190688 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4880ed-e2fa-42f5-8981-d94d7b30a4a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:08.193333 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4880ed-e2fa-42f5-8981-d94d7b30a4a9 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:08.227706 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1072/2139] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:09:08 2022] POST /identity/v3/users => generated 295 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:08.231639 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d70fc-d3d9-4e4c-8393-01163fa7a34a admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:08.249484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d70fc-d3d9-4e4c-8393-01163fa7a34a admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:08.254889 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cf9f0-be28-4531-ba0f-032db5c21c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:08.255850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cf9f0-be28-4531-ba0f-032db5c21c53 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:08.256132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cf9f0-be28-4531-ba0f-032db5c21c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:08.256408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cf9f0-be28-4531-ba0f-032db5c21c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:08.257254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1cf9f0-be28-4531-ba0f-032db5c21c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:08.258830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1cf9f0-be28-4531-ba0f-032db5c21c53 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:08.277009 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4880ed-e2fa-42f5-8981-d94d7b30a4a9 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:08.279587 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1068/2140] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:08.288648 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4880ed-e2fa-42f5-8981-d94d7b30a4a9 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:08.293382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc5b11-49fb-4f77-8d27-cc850734ef40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:08.294045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc5b11-49fb-4f77-8d27-cc850734ef40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:08.294327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc5b11-49fb-4f77-8d27-cc850734ef40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:08.294611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc5b11-49fb-4f77-8d27-cc850734ef40 admin admin] PATH_INFO: `/v3/projects/9d33f4f0839747c0af20c5d91b741407/users/b38dbf95c17d4e8ca2bc591514a852e4/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:08.311745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc5b11-49fb-4f77-8d27-cc850734ef40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d33f4f0839747c0af20c5d91b741407, user_id=b38dbf95c17d4e8ca2bc591514a852e4, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:08.314527 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc5b11-49fb-4f77-8d27-cc850734ef40 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:08.332006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1073/2141] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:08 2022] PUT /identity/v3/projects/9d33f4f0839747c0af20c5d91b741407/users/b38dbf95c17d4e8ca2bc591514a852e4/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:08.336650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cf9f0-be28-4531-ba0f-032db5c21c53 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:08.348974 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cf9f0-be28-4531-ba0f-032db5c21c53 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:08.412615 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2df15b-258d-4cba-b645-3aa81e838b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:08.413310 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2df15b-258d-4cba-b645-3aa81e838b13 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:08.413844 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2df15b-258d-4cba-b645-3aa81e838b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:08.413988 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2df15b-258d-4cba-b645-3aa81e838b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:08.414619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2df15b-258d-4cba-b645-3aa81e838b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:08.416625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2df15b-258d-4cba-b645-3aa81e838b13 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:08.426857 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1069/2142] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:08.432342 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc5b11-49fb-4f77-8d27-cc850734ef40 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:08.443572 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc5b11-49fb-4f77-8d27-cc850734ef40 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:08.449439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35715b1-58c0-4fcb-b63a-fe91092f79bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:08.450150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35715b1-58c0-4fcb-b63a-fe91092f79bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:08.450435 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35715b1-58c0-4fcb-b63a-fe91092f79bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:08.450701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35715b1-58c0-4fcb-b63a-fe91092f79bd admin admin] PATH_INFO: `/v3/projects/9d33f4f0839747c0af20c5d91b741407/users/b38dbf95c17d4e8ca2bc591514a852e4/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:08.452897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35715b1-58c0-4fcb-b63a-fe91092f79bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d33f4f0839747c0af20c5d91b741407, user_id=b38dbf95c17d4e8ca2bc591514a852e4, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:08.455690 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35715b1-58c0-4fcb-b63a-fe91092f79bd admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:08.470542 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1074/2143] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:08 2022] PUT /identity/v3/projects/9d33f4f0839747c0af20c5d91b741407/users/b38dbf95c17d4e8ca2bc591514a852e4/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:08.482422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53af47-71c0-4127-b79b-e0ba673f71d6 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:08.482791 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53af47-71c0-4127-b79b-e0ba673f71d6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:08.483126 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53af47-71c0-4127-b79b-e0ba673f71d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:08.516918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-cb53af47-71c0-4127-b79b-e0ba673f71d6 None None] MFA Rules not processed for user `b38dbf95c17d4e8ca2bc591514a852e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:08.518433 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-cb53af47-71c0-4127-b79b-e0ba673f71d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:08.596777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1070/2144] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:08.608919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35715b1-58c0-4fcb-b63a-fe91092f79bd admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:08.611066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c35715b1-58c0-4fcb-b63a-fe91092f79bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:08.717602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35715b1-58c0-4fcb-b63a-fe91092f79bd admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:08.721833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3951398-8db1-40ca-b0bf-0bb503e19ef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:08.722465 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3951398-8db1-40ca-b0bf-0bb503e19ef5 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:08.722741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3951398-8db1-40ca-b0bf-0bb503e19ef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:08.723012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3951398-8db1-40ca-b0bf-0bb503e19ef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:08.733405 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3951398-8db1-40ca-b0bf-0bb503e19ef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:08.734853 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3951398-8db1-40ca-b0bf-0bb503e19ef5 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:08.745529 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1075/2145] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:09:08.788086 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53af47-71c0-4127-b79b-e0ba673f71d6 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:08.797613 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53af47-71c0-4127-b79b-e0ba673f71d6 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:08.801176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cf67f-e15e-4e08-b4bb-d9c0dd36c921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:08.801777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cf67f-e15e-4e08-b4bb-d9c0dd36c921 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:08.802082 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cf67f-e15e-4e08-b4bb-d9c0dd36c921 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:08.802349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cf67f-e15e-4e08-b4bb-d9c0dd36c921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:08.811550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:08.860345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cf67f-e15e-4e08-b4bb-d9c0dd36c921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:08.861694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cf67f-e15e-4e08-b4bb-d9c0dd36c921 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:08.870780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1071/2146] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:09:12.045994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3951398-8db1-40ca-b0bf-0bb503e19ef5 service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:12.147586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3951398-8db1-40ca-b0bf-0bb503e19ef5 service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:12.152037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0babcb-82ba-46c0-9909-c9d58b4899ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:12.153164 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0babcb-82ba-46c0-9909-c9d58b4899ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:12.153777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0babcb-82ba-46c0-9909-c9d58b4899ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:12.154244 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0babcb-82ba-46c0-9909-c9d58b4899ac admin admin] PATH_INFO: `/v3/users/d4ea7c5a840b4048bb649812d07fc5cb` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:12.155652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0babcb-82ba-46c0-9909-c9d58b4899ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ea7c5a840b4048bb649812d07fc5cb)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:12.157825 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0babcb-82ba-46c0-9909-c9d58b4899ac admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:12.193098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:12.213330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8f0babcb-82ba-46c0-9909-c9d58b4899ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ea7c5a840b4048bb649812d07fc5cb'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:12.222266 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8f0babcb-82ba-46c0-9909-c9d58b4899ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ea7c5a840b4048bb649812d07fc5cb'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:12.223315 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8f0babcb-82ba-46c0-9909-c9d58b4899ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ea7c5a840b4048bb649812d07fc5cb'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:12.223998 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8f0babcb-82ba-46c0-9909-c9d58b4899ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ea7c5a840b4048bb649812d07fc5cb'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:12.231423 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8f0babcb-82ba-46c0-9909-c9d58b4899ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ea7c5a840b4048bb649812d07fc5cb'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:12.250301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1076/2147] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:12 2022] DELETE /identity/v3/users/d4ea7c5a840b4048bb649812d07fc5cb => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:12.254718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422dfcc6-5f3b-4f4c-9349-3296cb083b8d None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:12.255222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422dfcc6-5f3b-4f4c-9349-3296cb083b8d None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:12.255335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422dfcc6-5f3b-4f4c-9349-3296cb083b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:12.284470 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-422dfcc6-5f3b-4f4c-9349-3296cb083b8d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:12.285765 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-422dfcc6-5f3b-4f4c-9349-3296cb083b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:12.348213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1072/2148] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:12.358321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0babcb-82ba-46c0-9909-c9d58b4899ac admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:12.360022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-8f0babcb-82ba-46c0-9909-c9d58b4899ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:12.429300 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0babcb-82ba-46c0-9909-c9d58b4899ac admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:12.433062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61b51b-4604-4c1e-a298-1e9ef2a8d86a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:12.433683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61b51b-4604-4c1e-a298-1e9ef2a8d86a service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:12.433960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61b51b-4604-4c1e-a298-1e9ef2a8d86a service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:12.434228 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61b51b-4604-4c1e-a298-1e9ef2a8d86a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:12.444926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61b51b-4604-4c1e-a298-1e9ef2a8d86a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:12.446761 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61b51b-4604-4c1e-a298-1e9ef2a8d86a service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:12.457183 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1077/2149] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:09:12.733805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422dfcc6-5f3b-4f4c-9349-3296cb083b8d None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:12.735735 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-422dfcc6-5f3b-4f4c-9349-3296cb083b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:12.755744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422dfcc6-5f3b-4f4c-9349-3296cb083b8d None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:12.763961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418f81d-e255-41af-85d6-6289eb9576ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:12.764726 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418f81d-e255-41af-85d6-6289eb9576ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:12.765068 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418f81d-e255-41af-85d6-6289eb9576ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:12.765418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418f81d-e255-41af-85d6-6289eb9576ce admin admin] PATH_INFO: `/v3/projects/fdeafe2f60fe4bc68663e8f93437038e` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:12.766625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418f81d-e255-41af-85d6-6289eb9576ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdeafe2f60fe4bc68663e8f93437038e)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:12.768692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418f81d-e255-41af-85d6-6289eb9576ce admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:12.821087 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-4418f81d-e255-41af-85d6-6289eb9576ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdeafe2f60fe4bc68663e8f93437038e', 'request_id': 'req-4418f81d-e255-41af-85d6-6289eb9576ce'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:12.829224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-4418f81d-e255-41af-85d6-6289eb9576ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdeafe2f60fe4bc68663e8f93437038e', 'request_id': 'req-4418f81d-e255-41af-85d6-6289eb9576ce'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:12.834892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-4418f81d-e255-41af-85d6-6289eb9576ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdeafe2f60fe4bc68663e8f93437038e', 'request_id': 'req-4418f81d-e255-41af-85d6-6289eb9576ce'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:12.847954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-4418f81d-e255-41af-85d6-6289eb9576ce admin admin] The token cache is being invalidate because project fdeafe2f60fe4bc68663e8f93437038e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:12.848241 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-4418f81d-e255-41af-85d6-6289eb9576ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:12.848888 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-4418f81d-e255-41af-85d6-6289eb9576ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:12.851197 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1073/2150] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:12 2022] DELETE /identity/v3/projects/fdeafe2f60fe4bc68663e8f93437038e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:12.854431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61b51b-4604-4c1e-a298-1e9ef2a8d86a service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:12.863998 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61b51b-4604-4c1e-a298-1e9ef2a8d86a service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:12.930137 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33877bb-ef16-47c9-896f-f945f2e822f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:12.930809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33877bb-ef16-47c9-896f-f945f2e822f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:12.931087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33877bb-ef16-47c9-896f-f945f2e822f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:12.931353 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33877bb-ef16-47c9-896f-f945f2e822f3 admin admin] PATH_INFO: `/v3/users/2a5b0e85206a4dfaaa4b4634ad1c7a1d` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:12.932615 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33877bb-ef16-47c9-896f-f945f2e822f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a5b0e85206a4dfaaa4b4634ad1c7a1d)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:12.934553 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33877bb-ef16-47c9-896f-f945f2e822f3 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:12.985534 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a33877bb-ef16-47c9-896f-f945f2e822f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a5b0e85206a4dfaaa4b4634ad1c7a1d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:12.988747 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:12.994625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a33877bb-ef16-47c9-896f-f945f2e822f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a5b0e85206a4dfaaa4b4634ad1c7a1d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:12.995264 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a33877bb-ef16-47c9-896f-f945f2e822f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a5b0e85206a4dfaaa4b4634ad1c7a1d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:12.995989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a33877bb-ef16-47c9-896f-f945f2e822f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a5b0e85206a4dfaaa4b4634ad1c7a1d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:13.008535 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a33877bb-ef16-47c9-896f-f945f2e822f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a5b0e85206a4dfaaa4b4634ad1c7a1d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:13.030777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1078/2151] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:12 2022] DELETE /identity/v3/users/2a5b0e85206a4dfaaa4b4634ad1c7a1d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:13.570573 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418f81d-e255-41af-85d6-6289eb9576ce admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:13.572166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-4418f81d-e255-41af-85d6-6289eb9576ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:13.604694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418f81d-e255-41af-85d6-6289eb9576ce admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:13.658008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:13.686764 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6ab1f-e096-4514-a48a-b80828026f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:13.687550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6ab1f-e096-4514-a48a-b80828026f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:13.687893 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6ab1f-e096-4514-a48a-b80828026f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:13.688309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6ab1f-e096-4514-a48a-b80828026f17 admin admin] PATH_INFO: `/v3/projects/98081f3b934c4e6383ff7abb0f071975` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:13.689468 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6ab1f-e096-4514-a48a-b80828026f17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98081f3b934c4e6383ff7abb0f071975)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:13.691465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6ab1f-e096-4514-a48a-b80828026f17 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:13.771830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a1b6ab1f-e096-4514-a48a-b80828026f17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98081f3b934c4e6383ff7abb0f071975', 'request_id': 'req-a1b6ab1f-e096-4514-a48a-b80828026f17'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:13.782292 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a1b6ab1f-e096-4514-a48a-b80828026f17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98081f3b934c4e6383ff7abb0f071975', 'request_id': 'req-a1b6ab1f-e096-4514-a48a-b80828026f17'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:13.796271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a1b6ab1f-e096-4514-a48a-b80828026f17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98081f3b934c4e6383ff7abb0f071975', 'request_id': 'req-a1b6ab1f-e096-4514-a48a-b80828026f17'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:13.810675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a1b6ab1f-e096-4514-a48a-b80828026f17 admin admin] The token cache is being invalidate because project 98081f3b934c4e6383ff7abb0f071975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:13.810915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a1b6ab1f-e096-4514-a48a-b80828026f17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:13.811765 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a1b6ab1f-e096-4514-a48a-b80828026f17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:13.813385 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1074/2152] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:13 2022] DELETE /identity/v3/projects/98081f3b934c4e6383ff7abb0f071975 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:13.817729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33877bb-ef16-47c9-896f-f945f2e822f3 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:13.819603 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a33877bb-ef16-47c9-896f-f945f2e822f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:13.834185 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33877bb-ef16-47c9-896f-f945f2e822f3 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:13.897934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac5d87-33e3-46bd-8a7e-d0f4c6c3f79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:13.898667 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac5d87-33e3-46bd-8a7e-d0f4c6c3f79a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:13.898990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac5d87-33e3-46bd-8a7e-d0f4c6c3f79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:13.899406 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac5d87-33e3-46bd-8a7e-d0f4c6c3f79a admin admin] PATH_INFO: `/v3/users/b38dbf95c17d4e8ca2bc591514a852e4` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:13.901145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac5d87-33e3-46bd-8a7e-d0f4c6c3f79a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b38dbf95c17d4e8ca2bc591514a852e4)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:13.903383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac5d87-33e3-46bd-8a7e-d0f4c6c3f79a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:13.969920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e5ac5d87-33e3-46bd-8a7e-d0f4c6c3f79a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b38dbf95c17d4e8ca2bc591514a852e4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:13.980280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e5ac5d87-33e3-46bd-8a7e-d0f4c6c3f79a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b38dbf95c17d4e8ca2bc591514a852e4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:13.980965 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e5ac5d87-33e3-46bd-8a7e-d0f4c6c3f79a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b38dbf95c17d4e8ca2bc591514a852e4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:13.981608 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e5ac5d87-33e3-46bd-8a7e-d0f4c6c3f79a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b38dbf95c17d4e8ca2bc591514a852e4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:13.990758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e5ac5d87-33e3-46bd-8a7e-d0f4c6c3f79a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b38dbf95c17d4e8ca2bc591514a852e4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:14.010495 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1079/2153] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:13 2022] DELETE /identity/v3/users/b38dbf95c17d4e8ca2bc591514a852e4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:14.362205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6ab1f-e096-4514-a48a-b80828026f17 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:14.364213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a1b6ab1f-e096-4514-a48a-b80828026f17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:14.396293 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6ab1f-e096-4514-a48a-b80828026f17 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:14.462794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6eef62-5fe9-4b74-a646-e1fb920a0947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:14.463729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6eef62-5fe9-4b74-a646-e1fb920a0947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:14.464003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6eef62-5fe9-4b74-a646-e1fb920a0947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:14.464272 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6eef62-5fe9-4b74-a646-e1fb920a0947 admin admin] PATH_INFO: `/v3/projects/9d33f4f0839747c0af20c5d91b741407` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:14.465735 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6eef62-5fe9-4b74-a646-e1fb920a0947 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d33f4f0839747c0af20c5d91b741407)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:14.467766 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6eef62-5fe9-4b74-a646-e1fb920a0947 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:14.538271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-9d6eef62-5fe9-4b74-a646-e1fb920a0947 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d33f4f0839747c0af20c5d91b741407', 'request_id': 'req-9d6eef62-5fe9-4b74-a646-e1fb920a0947'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:14.570337 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-9d6eef62-5fe9-4b74-a646-e1fb920a0947 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d33f4f0839747c0af20c5d91b741407', 'request_id': 'req-9d6eef62-5fe9-4b74-a646-e1fb920a0947'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:14.583644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-9d6eef62-5fe9-4b74-a646-e1fb920a0947 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d33f4f0839747c0af20c5d91b741407', 'request_id': 'req-9d6eef62-5fe9-4b74-a646-e1fb920a0947'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:14.587467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:14.600421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-9d6eef62-5fe9-4b74-a646-e1fb920a0947 admin admin] The token cache is being invalidate because project 9d33f4f0839747c0af20c5d91b741407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:14.601111 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-9d6eef62-5fe9-4b74-a646-e1fb920a0947 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:14.602109 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-9d6eef62-5fe9-4b74-a646-e1fb920a0947 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:14.604837 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1075/2154] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:14 2022] DELETE /identity/v3/projects/9d33f4f0839747c0af20c5d91b741407 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:14.611486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54d370-322f-4174-a2d6-b8c325b1ed3f None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:14.611486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54d370-322f-4174-a2d6-b8c325b1ed3f None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:14.614071 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e54d370-322f-4174-a2d6-b8c325b1ed3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:14.636695 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-2e54d370-322f-4174-a2d6-b8c325b1ed3f None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:14.638283 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-2e54d370-322f-4174-a2d6-b8c325b1ed3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:14.721863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:14.754233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1080/2155] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:09:14.765781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782af33-74d0-41ca-8adf-a507fffcd327 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:14.766066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782af33-74d0-41ca-8adf-a507fffcd327 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:14.766337 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782af33-74d0-41ca-8adf-a507fffcd327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:14.801203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-3782af33-74d0-41ca-8adf-a507fffcd327 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:14.802482 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3782af33-74d0-41ca-8adf-a507fffcd327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:14.817270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1076/2156] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:14.818181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54d370-322f-4174-a2d6-b8c325b1ed3f None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:14.831006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e54d370-322f-4174-a2d6-b8c325b1ed3f None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:14.838345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbb2c1-cf19-4a32-a78f-a2ddd81c17de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:14.839719 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbb2c1-cf19-4a32-a78f-a2ddd81c17de admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:14.840246 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbb2c1-cf19-4a32-a78f-a2ddd81c17de admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:14.840584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbb2c1-cf19-4a32-a78f-a2ddd81c17de admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:14.841366 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fbb2c1-cf19-4a32-a78f-a2ddd81c17de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:09:14.841806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fbb2c1-cf19-4a32-a78f-a2ddd81c17de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:14.847008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fbb2c1-cf19-4a32-a78f-a2ddd81c17de admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:14.866289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1081/2157] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:09:14.870951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782af33-74d0-41ca-8adf-a507fffcd327 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:14.889048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3782af33-74d0-41ca-8adf-a507fffcd327 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:14.893995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20ac5b-6cea-4894-9e9d-f352b48fed4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:14.894614 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20ac5b-6cea-4894-9e9d-f352b48fed4a admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:14.894882 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20ac5b-6cea-4894-9e9d-f352b48fed4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:14.895304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20ac5b-6cea-4894-9e9d-f352b48fed4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:14.895979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20ac5b-6cea-4894-9e9d-f352b48fed4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:14.898063 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20ac5b-6cea-4894-9e9d-f352b48fed4a admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:14.934861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1077/2158] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:09:14 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:14.936499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbb2c1-cf19-4a32-a78f-a2ddd81c17de admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:14.948451 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbb2c1-cf19-4a32-a78f-a2ddd81c17de admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:15.015098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88ad20-8a60-4ebb-a2d5-9338627798a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:15.015899 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88ad20-8a60-4ebb-a2d5-9338627798a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:15.016242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88ad20-8a60-4ebb-a2d5-9338627798a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:15.016575 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88ad20-8a60-4ebb-a2d5-9338627798a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:15.017277 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88ad20-8a60-4ebb-a2d5-9338627798a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:15.019266 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a88ad20-8a60-4ebb-a2d5-9338627798a1 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:15.059105 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1082/2159] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:09:14 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:15.062754 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20ac5b-6cea-4894-9e9d-f352b48fed4a admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:15.075941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20ac5b-6cea-4894-9e9d-f352b48fed4a admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:15.081877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a545c4d-e911-4697-abc2-e0bf6d74e364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:15.082679 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a545c4d-e911-4697-abc2-e0bf6d74e364 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:15.083023 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a545c4d-e911-4697-abc2-e0bf6d74e364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:15.083380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a545c4d-e911-4697-abc2-e0bf6d74e364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:15.084294 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a545c4d-e911-4697-abc2-e0bf6d74e364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:15.086395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a545c4d-e911-4697-abc2-e0bf6d74e364 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:15.100406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1078/2160] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:15.105153 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88ad20-8a60-4ebb-a2d5-9338627798a1 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:15.121058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88ad20-8a60-4ebb-a2d5-9338627798a1 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:15.129287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f05fa28-6427-49a4-8270-ebc987f8f560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:15.130038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05fa28-6427-49a4-8270-ebc987f8f560 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:15.130387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05fa28-6427-49a4-8270-ebc987f8f560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:15.130789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f05fa28-6427-49a4-8270-ebc987f8f560 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:15.131598 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05fa28-6427-49a4-8270-ebc987f8f560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:15.133348 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f05fa28-6427-49a4-8270-ebc987f8f560 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:15.145207 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1083/2161] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:15.149151 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a545c4d-e911-4697-abc2-e0bf6d74e364 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:15.159876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a545c4d-e911-4697-abc2-e0bf6d74e364 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:15.166883 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa915f-bfa4-4909-b077-5474ea7665f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:15.167830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa915f-bfa4-4909-b077-5474ea7665f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:15.167952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa915f-bfa4-4909-b077-5474ea7665f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:15.169239 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa915f-bfa4-4909-b077-5474ea7665f6 admin admin] PATH_INFO: `/v3/projects/3fd1ead1d4f348fe9f2ba77522143755/users/824c45c2b32f45ffb7fcd603a6eeb343/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:15.189785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa915f-bfa4-4909-b077-5474ea7665f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd1ead1d4f348fe9f2ba77522143755, user_id=824c45c2b32f45ffb7fcd603a6eeb343, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:15.192527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa915f-bfa4-4909-b077-5474ea7665f6 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:15.222747 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1079/2162] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:15 2022] PUT /identity/v3/projects/3fd1ead1d4f348fe9f2ba77522143755/users/824c45c2b32f45ffb7fcd603a6eeb343/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:15.225343 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1374dc-8d9c-487c-9291-079c9a0f2ccb None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:15.225736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1374dc-8d9c-487c-9291-079c9a0f2ccb None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:15.226013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1374dc-8d9c-487c-9291-079c9a0f2ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:15.263791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-9d1374dc-8d9c-487c-9291-079c9a0f2ccb None None] MFA Rules not processed for user `824c45c2b32f45ffb7fcd603a6eeb343`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:15.265124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9d1374dc-8d9c-487c-9291-079c9a0f2ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:15.369544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1084/2163] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:15.370741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa915f-bfa4-4909-b077-5474ea7665f6 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:15.453158 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa915f-bfa4-4909-b077-5474ea7665f6 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:15.462637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1630a44-af22-430e-ac48-681eade8835c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:15.463391 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1630a44-af22-430e-ac48-681eade8835c admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:15.463686 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1630a44-af22-430e-ac48-681eade8835c admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:15.463967 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1630a44-af22-430e-ac48-681eade8835c admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:15.464586 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1630a44-af22-430e-ac48-681eade8835c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:15.466892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1630a44-af22-430e-ac48-681eade8835c admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:15.507842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1080/2164] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:09:15 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:15.511725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1374dc-8d9c-487c-9291-079c9a0f2ccb None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:15.527374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1374dc-8d9c-487c-9291-079c9a0f2ccb None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:15.596181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:15.655551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450cc40-f1e2-4c86-8c62-10899c8032e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:15.656301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450cc40-f1e2-4c86-8c62-10899c8032e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:15.656610 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450cc40-f1e2-4c86-8c62-10899c8032e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:15.656882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450cc40-f1e2-4c86-8c62-10899c8032e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:15.657571 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450cc40-f1e2-4c86-8c62-10899c8032e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:15.665112 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f450cc40-f1e2-4c86-8c62-10899c8032e3 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:15.720834 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1085/2165] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:09:15 2022] POST /identity/v3/users => generated 299 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:09:15.722634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1630a44-af22-430e-ac48-681eade8835c admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:15.741398 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1630a44-af22-430e-ac48-681eade8835c admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:15.752847 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf6036-4bb2-4601-a74d-db42c7caaa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:15.753590 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf6036-4bb2-4601-a74d-db42c7caaa62 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:15.753965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf6036-4bb2-4601-a74d-db42c7caaa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:15.754323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf6036-4bb2-4601-a74d-db42c7caaa62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:15.755208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf6036-4bb2-4601-a74d-db42c7caaa62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:15.758112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cf6036-4bb2-4601-a74d-db42c7caaa62 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:15.773736 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1081/2166] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:15.777362 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450cc40-f1e2-4c86-8c62-10899c8032e3 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:15.788418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f450cc40-f1e2-4c86-8c62-10899c8032e3 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:15.793634 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db172cd2-d887-4e6d-bd34-685898ebb9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:15.794399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db172cd2-d887-4e6d-bd34-685898ebb9b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:15.794774 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db172cd2-d887-4e6d-bd34-685898ebb9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:15.795143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db172cd2-d887-4e6d-bd34-685898ebb9b4 admin admin] PATH_INFO: `/v3/projects/4656c5f859394c81900f22d824fe76eb/users/a4dad97744694b6a86dd159ef0f60882/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:15.816241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db172cd2-d887-4e6d-bd34-685898ebb9b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4656c5f859394c81900f22d824fe76eb, user_id=a4dad97744694b6a86dd159ef0f60882, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:15.819268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db172cd2-d887-4e6d-bd34-685898ebb9b4 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:15.838489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1086/2167] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:15 2022] PUT /identity/v3/projects/4656c5f859394c81900f22d824fe76eb/users/a4dad97744694b6a86dd159ef0f60882/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:09:15.842569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf6036-4bb2-4601-a74d-db42c7caaa62 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:15.854622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf6036-4bb2-4601-a74d-db42c7caaa62 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:15.918690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6351c8b-0853-4b1b-98d1-601b4e81660d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:15.919760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6351c8b-0853-4b1b-98d1-601b4e81660d admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:15.920139 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6351c8b-0853-4b1b-98d1-601b4e81660d admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:15.920565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6351c8b-0853-4b1b-98d1-601b4e81660d admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:15.921859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6351c8b-0853-4b1b-98d1-601b4e81660d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:15.923125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6351c8b-0853-4b1b-98d1-601b4e81660d admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:15.933918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1082/2168] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:09:15.939005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db172cd2-d887-4e6d-bd34-685898ebb9b4 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:15.939934 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6351c8b-0853-4b1b-98d1-601b4e81660d admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:15.941801 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b6351c8b-0853-4b1b-98d1-601b4e81660d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:15.949680 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db172cd2-d887-4e6d-bd34-685898ebb9b4 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:15.950276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:15.955932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6351c8b-0853-4b1b-98d1-601b4e81660d admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:15.958543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600885b0-ec44-43bc-adf3-35b62ee3ac1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:15.959350 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600885b0-ec44-43bc-adf3-35b62ee3ac1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:15.959805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600885b0-ec44-43bc-adf3-35b62ee3ac1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:15.960090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600885b0-ec44-43bc-adf3-35b62ee3ac1a admin admin] PATH_INFO: `/v3/projects/4656c5f859394c81900f22d824fe76eb/users/a4dad97744694b6a86dd159ef0f60882/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:15.961991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874cd5f9-a387-4961-9dec-cf36e301fc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:15.962158 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600885b0-ec44-43bc-adf3-35b62ee3ac1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4656c5f859394c81900f22d824fe76eb, user_id=a4dad97744694b6a86dd159ef0f60882, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:15.962625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874cd5f9-a387-4961-9dec-cf36e301fc91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:15.962920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874cd5f9-a387-4961-9dec-cf36e301fc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:15.963226 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874cd5f9-a387-4961-9dec-cf36e301fc91 admin admin] PATH_INFO: `/v3/users/675b1a5759844cdaa7e0bf071a3ee030` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:15.964386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874cd5f9-a387-4961-9dec-cf36e301fc91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=675b1a5759844cdaa7e0bf071a3ee030)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:15.964991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600885b0-ec44-43bc-adf3-35b62ee3ac1a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:15.966339 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874cd5f9-a387-4961-9dec-cf36e301fc91 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:15.976068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1087/2169] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:15 2022] PUT /identity/v3/projects/4656c5f859394c81900f22d824fe76eb/users/a4dad97744694b6a86dd159ef0f60882/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:09:15.986916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562b70f-b9b0-4fc7-be19-715f10a7c89a None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:15.987296 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562b70f-b9b0-4fc7-be19-715f10a7c89a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:15.987739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7562b70f-b9b0-4fc7-be19-715f10a7c89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:16.017831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-7562b70f-b9b0-4fc7-be19-715f10a7c89a None None] MFA Rules not processed for user `a4dad97744694b6a86dd159ef0f60882`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:16.019058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-7562b70f-b9b0-4fc7-be19-715f10a7c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:16.025581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-874cd5f9-a387-4961-9dec-cf36e301fc91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '675b1a5759844cdaa7e0bf071a3ee030'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:16.033823 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-874cd5f9-a387-4961-9dec-cf36e301fc91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '675b1a5759844cdaa7e0bf071a3ee030'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:16.034695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-874cd5f9-a387-4961-9dec-cf36e301fc91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '675b1a5759844cdaa7e0bf071a3ee030'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:16.035381 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-874cd5f9-a387-4961-9dec-cf36e301fc91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '675b1a5759844cdaa7e0bf071a3ee030'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:16.048724 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-874cd5f9-a387-4961-9dec-cf36e301fc91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '675b1a5759844cdaa7e0bf071a3ee030'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:16.061289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1083/2170] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:15 2022] DELETE /identity/v3/users/675b1a5759844cdaa7e0bf071a3ee030 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:09:16.097186 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1088/2171] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:09:16.109449 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874cd5f9-a387-4961-9dec-cf36e301fc91 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:16.111167 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-874cd5f9-a387-4961-9dec-cf36e301fc91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:16.181641 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874cd5f9-a387-4961-9dec-cf36e301fc91 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:16.185520 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17482668-8603-4eef-9d50-ac95490aa5c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:16.186218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17482668-8603-4eef-9d50-ac95490aa5c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:16.186551 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17482668-8603-4eef-9d50-ac95490aa5c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:16.186886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17482668-8603-4eef-9d50-ac95490aa5c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:16.248643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562b70f-b9b0-4fc7-be19-715f10a7c89a None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:16.250192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-7562b70f-b9b0-4fc7-be19-715f10a7c89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:16.263295 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17482668-8603-4eef-9d50-ac95490aa5c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:16.264784 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17482668-8603-4eef-9d50-ac95490aa5c3 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:16.274975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1084/2172] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:09:16.323127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7562b70f-b9b0-4fc7-be19-715f10a7c89a None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:16.352092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:16.386458 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36573bb-01d2-4b9a-aa42-3aaaba042551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:16.387107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36573bb-01d2-4b9a-aa42-3aaaba042551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:16.387385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36573bb-01d2-4b9a-aa42-3aaaba042551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:16.387688 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36573bb-01d2-4b9a-aa42-3aaaba042551 admin admin] PATH_INFO: `/v3/projects/be0b2aab8dbf41a7a3c590ca61f022f8` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:16.388817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36573bb-01d2-4b9a-aa42-3aaaba042551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be0b2aab8dbf41a7a3c590ca61f022f8)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:16.391084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36573bb-01d2-4b9a-aa42-3aaaba042551 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:16.451973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b36573bb-01d2-4b9a-aa42-3aaaba042551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be0b2aab8dbf41a7a3c590ca61f022f8', 'request_id': 'req-b36573bb-01d2-4b9a-aa42-3aaaba042551'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:16.460410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b36573bb-01d2-4b9a-aa42-3aaaba042551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be0b2aab8dbf41a7a3c590ca61f022f8', 'request_id': 'req-b36573bb-01d2-4b9a-aa42-3aaaba042551'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:16.466700 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b36573bb-01d2-4b9a-aa42-3aaaba042551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be0b2aab8dbf41a7a3c590ca61f022f8', 'request_id': 'req-b36573bb-01d2-4b9a-aa42-3aaaba042551'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:16.479021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b36573bb-01d2-4b9a-aa42-3aaaba042551 admin admin] The token cache is being invalidate because project be0b2aab8dbf41a7a3c590ca61f022f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:16.479304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b36573bb-01d2-4b9a-aa42-3aaaba042551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:16.479981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b36573bb-01d2-4b9a-aa42-3aaaba042551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:16.481490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1089/2173] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:16 2022] DELETE /identity/v3/projects/be0b2aab8dbf41a7a3c590ca61f022f8 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:09:16.488326 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efcdb1-9e74-4ee7-9f3a-9bdaa6255f15 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:16.488592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efcdb1-9e74-4ee7-9f3a-9bdaa6255f15 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:16.488831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efcdb1-9e74-4ee7-9f3a-9bdaa6255f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:16.508480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-50efcdb1-9e74-4ee7-9f3a-9bdaa6255f15 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:16.509683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-50efcdb1-9e74-4ee7-9f3a-9bdaa6255f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:16.528877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:16.578199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1085/2174] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:16.586234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e292b-eab6-43e6-97bb-ff9635fe41da None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:16.586540 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e292b-eab6-43e6-97bb-ff9635fe41da None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:16.586813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e292b-eab6-43e6-97bb-ff9635fe41da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:16.617221 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-ec8e292b-eab6-43e6-97bb-ff9635fe41da None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:16.618459 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ec8e292b-eab6-43e6-97bb-ff9635fe41da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:16.626643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1090/2175] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:16.632249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efcdb1-9e74-4ee7-9f3a-9bdaa6255f15 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:16.642117 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efcdb1-9e74-4ee7-9f3a-9bdaa6255f15 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:16.646840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248541e7-5759-4d60-8005-ef5f0b4d9871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:16.647609 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248541e7-5759-4d60-8005-ef5f0b4d9871 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:16.647757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248541e7-5759-4d60-8005-ef5f0b4d9871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:16.648013 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248541e7-5759-4d60-8005-ef5f0b4d9871 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:16.648647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248541e7-5759-4d60-8005-ef5f0b4d9871 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:09:16.648967 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248541e7-5759-4d60-8005-ef5f0b4d9871 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:16.650787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248541e7-5759-4d60-8005-ef5f0b4d9871 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:16.663483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1086/2176] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:09:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:09:16.667797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e292b-eab6-43e6-97bb-ff9635fe41da None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:16.679355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e292b-eab6-43e6-97bb-ff9635fe41da None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:16.684384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65358961-b300-4de3-b6af-4e433c0f4dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:16.684997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65358961-b300-4de3-b6af-4e433c0f4dba admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:16.685293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65358961-b300-4de3-b6af-4e433c0f4dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:16.685564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65358961-b300-4de3-b6af-4e433c0f4dba admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:16.686200 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65358961-b300-4de3-b6af-4e433c0f4dba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:16.688153 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65358961-b300-4de3-b6af-4e433c0f4dba admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:16.719487 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1091/2177] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:09:16 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:09:16.723680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248541e7-5759-4d60-8005-ef5f0b4d9871 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:16.738120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248541e7-5759-4d60-8005-ef5f0b4d9871 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:16.801011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749b87c-4495-4d91-8465-305bf0289b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:16.801651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749b87c-4495-4d91-8465-305bf0289b9f admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:16.801922 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749b87c-4495-4d91-8465-305bf0289b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:16.802189 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749b87c-4495-4d91-8465-305bf0289b9f admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:16.802835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749b87c-4495-4d91-8465-305bf0289b9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:16.804790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1749b87c-4495-4d91-8465-305bf0289b9f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:16.830508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1087/2178] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:09:16 2022] POST /identity/v3/users => generated 286 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:16.834811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65358961-b300-4de3-b6af-4e433c0f4dba admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:16.845555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65358961-b300-4de3-b6af-4e433c0f4dba admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:16.850271 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6512f-3193-4937-a411-e1a0907baa4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:16.850876 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6512f-3193-4937-a411-e1a0907baa4b admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:16.851150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6512f-3193-4937-a411-e1a0907baa4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:16.851416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6512f-3193-4937-a411-e1a0907baa4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:16.852085 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6512f-3193-4937-a411-e1a0907baa4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:16.853719 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6512f-3193-4937-a411-e1a0907baa4b admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:16.866927 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1092/2179] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:16.868137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749b87c-4495-4d91-8465-305bf0289b9f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:16.878130 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749b87c-4495-4d91-8465-305bf0289b9f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:16.882930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a21cf-1b3a-441b-af33-931a67d35de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:16.883574 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a21cf-1b3a-441b-af33-931a67d35de9 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:16.883913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a21cf-1b3a-441b-af33-931a67d35de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:16.884186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a21cf-1b3a-441b-af33-931a67d35de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:16.884856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a21cf-1b3a-441b-af33-931a67d35de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:16.886465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a21cf-1b3a-441b-af33-931a67d35de9 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:16.897840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1088/2180] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:09:16.902784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6512f-3193-4937-a411-e1a0907baa4b admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:16.908905 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a21cf-1b3a-441b-af33-931a67d35de9 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:16.912600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-383a21cf-1b3a-441b-af33-931a67d35de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:16.915123 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6512f-3193-4937-a411-e1a0907baa4b admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:16.919492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff6d53b-b6cd-48fb-8b14-e87d49c387ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:16.920163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6d53b-b6cd-48fb-8b14-e87d49c387ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:16.920437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6d53b-b6cd-48fb-8b14-e87d49c387ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:16.920704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff6d53b-b6cd-48fb-8b14-e87d49c387ee admin admin] PATH_INFO: `/v3/projects/c67853e0ad324ac595650e59f9a90f56/users/965cbbdb06c248698ef54527c29767df/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:16.927255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a21cf-1b3a-441b-af33-931a67d35de9 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:16.932611 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcd9c7-d124-472e-b121-5cbe130bb5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:16.933385 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcd9c7-d124-472e-b121-5cbe130bb5b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:16.933743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcd9c7-d124-472e-b121-5cbe130bb5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:16.933943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcd9c7-d124-472e-b121-5cbe130bb5b3 admin admin] PATH_INFO: `/v3/users/824c45c2b32f45ffb7fcd603a6eeb343` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:16.935191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcd9c7-d124-472e-b121-5cbe130bb5b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=824c45c2b32f45ffb7fcd603a6eeb343)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:16.936767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6d53b-b6cd-48fb-8b14-e87d49c387ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=c67853e0ad324ac595650e59f9a90f56, user_id=965cbbdb06c248698ef54527c29767df, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:16.937288 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcd9c7-d124-472e-b121-5cbe130bb5b3 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:16.939910 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff6d53b-b6cd-48fb-8b14-e87d49c387ee admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:16.957038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1093/2181] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:16 2022] PUT /identity/v3/projects/c67853e0ad324ac595650e59f9a90f56/users/965cbbdb06c248698ef54527c29767df/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:09:16.962247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cf790-0b6f-421c-8a1e-c648a1c752d7 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:16.962614 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cf790-0b6f-421c-8a1e-c648a1c752d7 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:16.963001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cf790-0b6f-421c-8a1e-c648a1c752d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:16.983767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-6d2cf790-0b6f-421c-8a1e-c648a1c752d7 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:16.985065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6d2cf790-0b6f-421c-8a1e-c648a1c752d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:16.999275 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-88fcd9c7-d124-472e-b121-5cbe130bb5b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '824c45c2b32f45ffb7fcd603a6eeb343'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:17.008893 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-88fcd9c7-d124-472e-b121-5cbe130bb5b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '824c45c2b32f45ffb7fcd603a6eeb343'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:17.009536 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-88fcd9c7-d124-472e-b121-5cbe130bb5b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '824c45c2b32f45ffb7fcd603a6eeb343'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:17.010225 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-88fcd9c7-d124-472e-b121-5cbe130bb5b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '824c45c2b32f45ffb7fcd603a6eeb343'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:17.024818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-88fcd9c7-d124-472e-b121-5cbe130bb5b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '824c45c2b32f45ffb7fcd603a6eeb343'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:17.038645 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:17.038953 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1089/2182] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:16 2022] DELETE /identity/v3/users/824c45c2b32f45ffb7fcd603a6eeb343 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:17.043514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3701c-43d6-4063-8c1a-a5554b10c241 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:17.043808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3701c-43d6-4063-8c1a-a5554b10c241 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:17.044074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3701c-43d6-4063-8c1a-a5554b10c241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:17.051593 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:17.056828 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1094/2183] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:09:17.067384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cf790-0b6f-421c-8a1e-c648a1c752d7 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:17.069073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6d2cf790-0b6f-421c-8a1e-c648a1c752d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:17.076994 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-11f3701c-43d6-4063-8c1a-a5554b10c241 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:17.076994 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-11f3701c-43d6-4063-8c1a-a5554b10c241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:17.139211 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cf790-0b6f-421c-8a1e-c648a1c752d7 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:17.153249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1090/2184] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:09:17.161954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3701c-43d6-4063-8c1a-a5554b10c241 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:17.217118 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013fa26-7d13-40b7-b9b7-0983f4c6cb11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:17.217883 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013fa26-7d13-40b7-b9b7-0983f4c6cb11 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:17.218174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013fa26-7d13-40b7-b9b7-0983f4c6cb11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:17.218432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013fa26-7d13-40b7-b9b7-0983f4c6cb11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:17.233555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013fa26-7d13-40b7-b9b7-0983f4c6cb11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:17.235021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013fa26-7d13-40b7-b9b7-0983f4c6cb11 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:17.238908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3701c-43d6-4063-8c1a-a5554b10c241 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:17.242910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fa5d7-d2ab-4694-9658-0f3382bbaa37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:17.243560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fa5d7-d2ab-4694-9658-0f3382bbaa37 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:17.243835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fa5d7-d2ab-4694-9658-0f3382bbaa37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:17.244163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6fa5d7-d2ab-4694-9658-0f3382bbaa37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:17.245384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1095/2185] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:09:17.255525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fa5d7-d2ab-4694-9658-0f3382bbaa37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:17.257017 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6fa5d7-d2ab-4694-9658-0f3382bbaa37 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:17.267177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1091/2186] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:09:17.875361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013fa26-7d13-40b7-b9b7-0983f4c6cb11 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:17.877261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4013fa26-7d13-40b7-b9b7-0983f4c6cb11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:17.893706 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013fa26-7d13-40b7-b9b7-0983f4c6cb11 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:17.900182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8409a-d4bd-45dc-ac16-50914c0065e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:17.901256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8409a-d4bd-45dc-ac16-50914c0065e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:17.901567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8409a-d4bd-45dc-ac16-50914c0065e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:17.901846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8409a-d4bd-45dc-ac16-50914c0065e4 admin admin] PATH_INFO: `/v3/projects/3fd1ead1d4f348fe9f2ba77522143755` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:17.906659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8409a-d4bd-45dc-ac16-50914c0065e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fd1ead1d4f348fe9f2ba77522143755)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:17.908782 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8409a-d4bd-45dc-ac16-50914c0065e4 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:17.974889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-12c8409a-d4bd-45dc-ac16-50914c0065e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fd1ead1d4f348fe9f2ba77522143755', 'request_id': 'req-12c8409a-d4bd-45dc-ac16-50914c0065e4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:17.985934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-12c8409a-d4bd-45dc-ac16-50914c0065e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fd1ead1d4f348fe9f2ba77522143755', 'request_id': 'req-12c8409a-d4bd-45dc-ac16-50914c0065e4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:17.993040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-12c8409a-d4bd-45dc-ac16-50914c0065e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fd1ead1d4f348fe9f2ba77522143755', 'request_id': 'req-12c8409a-d4bd-45dc-ac16-50914c0065e4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:18.005007 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-12c8409a-d4bd-45dc-ac16-50914c0065e4 admin admin] The token cache is being invalidate because project 3fd1ead1d4f348fe9f2ba77522143755 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:18.005295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-12c8409a-d4bd-45dc-ac16-50914c0065e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:18.006073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-12c8409a-d4bd-45dc-ac16-50914c0065e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:18.007725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1096/2187] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:17 2022] DELETE /identity/v3/projects/3fd1ead1d4f348fe9f2ba77522143755 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:18.011851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fa5d7-d2ab-4694-9658-0f3382bbaa37 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:18.022263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6fa5d7-d2ab-4694-9658-0f3382bbaa37 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:18.082046 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fc795-a0e7-4003-9ad1-70d557ce5660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:18.082692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fc795-a0e7-4003-9ad1-70d557ce5660 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:18.082950 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fc795-a0e7-4003-9ad1-70d557ce5660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:18.083223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fc795-a0e7-4003-9ad1-70d557ce5660 admin admin] PATH_INFO: `/v3/users/a4dad97744694b6a86dd159ef0f60882` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:18.084392 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fc795-a0e7-4003-9ad1-70d557ce5660 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4dad97744694b6a86dd159ef0f60882)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:18.086345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7fc795-a0e7-4003-9ad1-70d557ce5660 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:18.152317 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3c7fc795-a0e7-4003-9ad1-70d557ce5660 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4dad97744694b6a86dd159ef0f60882'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:18.155715 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:18.162714 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3c7fc795-a0e7-4003-9ad1-70d557ce5660 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4dad97744694b6a86dd159ef0f60882'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:18.164550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3c7fc795-a0e7-4003-9ad1-70d557ce5660 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4dad97744694b6a86dd159ef0f60882'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:18.166132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3c7fc795-a0e7-4003-9ad1-70d557ce5660 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4dad97744694b6a86dd159ef0f60882'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:18.183435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3c7fc795-a0e7-4003-9ad1-70d557ce5660 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4dad97744694b6a86dd159ef0f60882'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:18.211077 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1092/2188] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:18 2022] DELETE /identity/v3/users/a4dad97744694b6a86dd159ef0f60882 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:18.698417 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8409a-d4bd-45dc-ac16-50914c0065e4 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:18.700205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-12c8409a-d4bd-45dc-ac16-50914c0065e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:18.728205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8409a-d4bd-45dc-ac16-50914c0065e4 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:18.782960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:18.817559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78acc9d-402d-487b-b595-e9e6d68dc5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:18.818214 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78acc9d-402d-487b-b595-e9e6d68dc5b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:18.818492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78acc9d-402d-487b-b595-e9e6d68dc5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:18.819310 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78acc9d-402d-487b-b595-e9e6d68dc5b2 admin admin] PATH_INFO: `/v3/projects/4656c5f859394c81900f22d824fe76eb` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:18.820282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78acc9d-402d-487b-b595-e9e6d68dc5b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4656c5f859394c81900f22d824fe76eb)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:18.822279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78acc9d-402d-487b-b595-e9e6d68dc5b2 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:18.875518 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e78acc9d-402d-487b-b595-e9e6d68dc5b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4656c5f859394c81900f22d824fe76eb', 'request_id': 'req-e78acc9d-402d-487b-b595-e9e6d68dc5b2'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:18.884735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e78acc9d-402d-487b-b595-e9e6d68dc5b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4656c5f859394c81900f22d824fe76eb', 'request_id': 'req-e78acc9d-402d-487b-b595-e9e6d68dc5b2'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:18.891666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e78acc9d-402d-487b-b595-e9e6d68dc5b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4656c5f859394c81900f22d824fe76eb', 'request_id': 'req-e78acc9d-402d-487b-b595-e9e6d68dc5b2'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:18.905692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e78acc9d-402d-487b-b595-e9e6d68dc5b2 admin admin] The token cache is being invalidate because project 4656c5f859394c81900f22d824fe76eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:18.905975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e78acc9d-402d-487b-b595-e9e6d68dc5b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:18.906926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e78acc9d-402d-487b-b595-e9e6d68dc5b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:18.908635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1097/2189] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:18 2022] DELETE /identity/v3/projects/4656c5f859394c81900f22d824fe76eb => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:18.913617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87e49a-5815-4d36-851b-ca366d516e7a None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:18.914206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87e49a-5815-4d36-851b-ca366d516e7a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:18.914206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87e49a-5815-4d36-851b-ca366d516e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:18.934963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-ee87e49a-5815-4d36-851b-ca366d516e7a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:18.936222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ee87e49a-5815-4d36-851b-ca366d516e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:19.043827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1093/2190] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:19.051934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68924fec-797e-446a-af96-0aab0ec9dd87 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:19.052215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68924fec-797e-446a-af96-0aab0ec9dd87 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:19.052484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68924fec-797e-446a-af96-0aab0ec9dd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:19.075060 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-68924fec-797e-446a-af96-0aab0ec9dd87 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:19.076226 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-68924fec-797e-446a-af96-0aab0ec9dd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:19.088298 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1098/2191] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:09:19.089581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87e49a-5815-4d36-851b-ca366d516e7a None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:19.101598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87e49a-5815-4d36-851b-ca366d516e7a None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:19.107418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0fd20-91e2-4e1e-a34e-9900f0b21d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:19.108156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0fd20-91e2-4e1e-a34e-9900f0b21d78 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:19.108509 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0fd20-91e2-4e1e-a34e-9900f0b21d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:19.108792 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0fd20-91e2-4e1e-a34e-9900f0b21d78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:19.110584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0fd20-91e2-4e1e-a34e-9900f0b21d78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:09:19.110714 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0fd20-91e2-4e1e-a34e-9900f0b21d78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:19.112429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0fd20-91e2-4e1e-a34e-9900f0b21d78 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:19.128172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1094/2192] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:09:19.132725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68924fec-797e-446a-af96-0aab0ec9dd87 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:19.142706 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68924fec-797e-446a-af96-0aab0ec9dd87 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:19.147669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b4b1f-570d-4a7e-b92c-3bff5b76a192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:19.148313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b4b1f-570d-4a7e-b92c-3bff5b76a192 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:19.148585 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b4b1f-570d-4a7e-b92c-3bff5b76a192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:19.148848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72b4b1f-570d-4a7e-b92c-3bff5b76a192 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:19.149494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b4b1f-570d-4a7e-b92c-3bff5b76a192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:19.169233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72b4b1f-570d-4a7e-b92c-3bff5b76a192 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:19.214734 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1099/2193] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:09:19 2022] POST /identity/v3/projects => generated 379 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:19.219208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0fd20-91e2-4e1e-a34e-9900f0b21d78 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:19.241645 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0fd20-91e2-4e1e-a34e-9900f0b21d78 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:19.324448 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfce5a-addd-4a98-a01b-a5fd6509a595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:19.325170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfce5a-addd-4a98-a01b-a5fd6509a595 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:19.325391 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfce5a-addd-4a98-a01b-a5fd6509a595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:19.325659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfce5a-addd-4a98-a01b-a5fd6509a595 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:19.326349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfce5a-addd-4a98-a01b-a5fd6509a595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:19.328330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfce5a-addd-4a98-a01b-a5fd6509a595 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:19.360118 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1095/2194] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:09:19 2022] POST /identity/v3/users => generated 290 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:19.364249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b4b1f-570d-4a7e-b92c-3bff5b76a192 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:19.392480 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72b4b1f-570d-4a7e-b92c-3bff5b76a192 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:19.398257 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8093f-f37a-4a15-aa21-3db033f28da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:19.399765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8093f-f37a-4a15-aa21-3db033f28da9 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:19.400299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8093f-f37a-4a15-aa21-3db033f28da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:19.400994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa8093f-f37a-4a15-aa21-3db033f28da9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:19.401854 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8093f-f37a-4a15-aa21-3db033f28da9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:19.405231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa8093f-f37a-4a15-aa21-3db033f28da9 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:19.417820 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1100/2195] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:19.422204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfce5a-addd-4a98-a01b-a5fd6509a595 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:19.438490 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfce5a-addd-4a98-a01b-a5fd6509a595 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:19.446375 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85f6e1-4431-4394-9f8b-53fa46d5ede4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:19.454013 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85f6e1-4431-4394-9f8b-53fa46d5ede4 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:19.454013 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85f6e1-4431-4394-9f8b-53fa46d5ede4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:19.459790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85f6e1-4431-4394-9f8b-53fa46d5ede4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:19.460965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85f6e1-4431-4394-9f8b-53fa46d5ede4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:19.465141 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d85f6e1-4431-4394-9f8b-53fa46d5ede4 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:19.478097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1096/2196] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:19.482657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8093f-f37a-4a15-aa21-3db033f28da9 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:19.496682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa8093f-f37a-4a15-aa21-3db033f28da9 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:19.515778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19211c73-2610-434f-8a73-c87104c6db9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:19.516649 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19211c73-2610-434f-8a73-c87104c6db9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:19.518085 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19211c73-2610-434f-8a73-c87104c6db9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:19.518700 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19211c73-2610-434f-8a73-c87104c6db9a admin admin] PATH_INFO: `/v3/projects/997959fa277e47fc84f726fd9ad2ca94/users/3047e947bd694148955b4571df2cec62/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:19.539618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19211c73-2610-434f-8a73-c87104c6db9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=997959fa277e47fc84f726fd9ad2ca94, user_id=3047e947bd694148955b4571df2cec62, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:19.546161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19211c73-2610-434f-8a73-c87104c6db9a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:19.567622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1101/2197] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:19 2022] PUT /identity/v3/projects/997959fa277e47fc84f726fd9ad2ca94/users/3047e947bd694148955b4571df2cec62/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:09:19.577570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22fbc6-d554-4ec5-8776-73b0ae8e5f3f None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:19.577880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22fbc6-d554-4ec5-8776-73b0ae8e5f3f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:19.578156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22fbc6-d554-4ec5-8776-73b0ae8e5f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:19.619554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-3d22fbc6-d554-4ec5-8776-73b0ae8e5f3f None None] MFA Rules not processed for user `3047e947bd694148955b4571df2cec62`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:19.620827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3d22fbc6-d554-4ec5-8776-73b0ae8e5f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:19.668461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:19.733224 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19211c73-2610-434f-8a73-c87104c6db9a admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:19.733897 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1097/2198] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:19.803751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19211c73-2610-434f-8a73-c87104c6db9a admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:19.809249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e68429-a7bb-478c-8398-35b7519a4566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:19.809896 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e68429-a7bb-478c-8398-35b7519a4566 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:19.810176 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e68429-a7bb-478c-8398-35b7519a4566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:19.810446 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e68429-a7bb-478c-8398-35b7519a4566 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:19.811104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e68429-a7bb-478c-8398-35b7519a4566 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:19.813128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e68429-a7bb-478c-8398-35b7519a4566 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:19.823235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:19.870765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1102/2199] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:09:19 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:09:19.872501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22fbc6-d554-4ec5-8776-73b0ae8e5f3f None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:19.906341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22fbc6-d554-4ec5-8776-73b0ae8e5f3f None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:20.049519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157aeb59-9d3e-4d61-8351-71f0bf96d0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:20.050141 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157aeb59-9d3e-4d61-8351-71f0bf96d0dc admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:20.050600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157aeb59-9d3e-4d61-8351-71f0bf96d0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:20.051012 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157aeb59-9d3e-4d61-8351-71f0bf96d0dc admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:20.051832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157aeb59-9d3e-4d61-8351-71f0bf96d0dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:20.053994 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157aeb59-9d3e-4d61-8351-71f0bf96d0dc admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:20.100337 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1098/2200] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:09:19 2022] POST /identity/v3/users => generated 296 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:20.104548 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e68429-a7bb-478c-8398-35b7519a4566 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:20.119365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e68429-a7bb-478c-8398-35b7519a4566 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:20.125758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add46acb-42a6-4e92-8e4d-8dd8eeec193b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:20.126526 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add46acb-42a6-4e92-8e4d-8dd8eeec193b admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:20.126892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add46acb-42a6-4e92-8e4d-8dd8eeec193b admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:20.127257 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add46acb-42a6-4e92-8e4d-8dd8eeec193b admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:20.128042 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add46acb-42a6-4e92-8e4d-8dd8eeec193b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:20.130131 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add46acb-42a6-4e92-8e4d-8dd8eeec193b admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:20.144919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1103/2201] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:09:20.149073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157aeb59-9d3e-4d61-8351-71f0bf96d0dc admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:20.161523 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157aeb59-9d3e-4d61-8351-71f0bf96d0dc admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:20.170062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aeaaa8-2b8d-456c-a1b5-c4264a563dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:20.170062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aeaaa8-2b8d-456c-a1b5-c4264a563dff admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:20.170062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aeaaa8-2b8d-456c-a1b5-c4264a563dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:20.171889 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aeaaa8-2b8d-456c-a1b5-c4264a563dff admin admin] PATH_INFO: `/v3/projects/fe191b8f63b1480d85416fc57c500c2d/users/1131cd25fcc644c284ab11fbce35e5d6/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:20.192957 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aeaaa8-2b8d-456c-a1b5-c4264a563dff admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe191b8f63b1480d85416fc57c500c2d, user_id=1131cd25fcc644c284ab11fbce35e5d6, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:20.196176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aeaaa8-2b8d-456c-a1b5-c4264a563dff admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:20.220725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1099/2202] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:20 2022] PUT /identity/v3/projects/fe191b8f63b1480d85416fc57c500c2d/users/1131cd25fcc644c284ab11fbce35e5d6/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:20.224829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add46acb-42a6-4e92-8e4d-8dd8eeec193b admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:20.238527 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add46acb-42a6-4e92-8e4d-8dd8eeec193b admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:20.331985 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f55ec-2fab-4f5e-b5ba-0963fd92d1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:20.332777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f55ec-2fab-4f5e-b5ba-0963fd92d1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:20.333052 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f55ec-2fab-4f5e-b5ba-0963fd92d1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:20.333318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f55ec-2fab-4f5e-b5ba-0963fd92d1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:20.333971 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f55ec-2fab-4f5e-b5ba-0963fd92d1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:20.335642 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f55ec-2fab-4f5e-b5ba-0963fd92d1a9 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:20.351698 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1104/2203] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:20.356047 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aeaaa8-2b8d-456c-a1b5-c4264a563dff admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:20.374455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aeaaa8-2b8d-456c-a1b5-c4264a563dff admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:20.380079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7adc2-a41e-45bf-9f20-50a74933f3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:20.380863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7adc2-a41e-45bf-9f20-50a74933f3af admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:20.381213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7adc2-a41e-45bf-9f20-50a74933f3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:20.381557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7adc2-a41e-45bf-9f20-50a74933f3af admin admin] PATH_INFO: `/v3/projects/fe191b8f63b1480d85416fc57c500c2d/users/1131cd25fcc644c284ab11fbce35e5d6/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:20.383750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7adc2-a41e-45bf-9f20-50a74933f3af admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe191b8f63b1480d85416fc57c500c2d, user_id=1131cd25fcc644c284ab11fbce35e5d6, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:20.387151 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7adc2-a41e-45bf-9f20-50a74933f3af admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:20.399963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1100/2204] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:20 2022] PUT /identity/v3/projects/fe191b8f63b1480d85416fc57c500c2d/users/1131cd25fcc644c284ab11fbce35e5d6/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:20.412624 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea363ef0-04ab-40ef-b698-5398418e7729 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:20.413142 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea363ef0-04ab-40ef-b698-5398418e7729 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:20.413564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea363ef0-04ab-40ef-b698-5398418e7729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:20.450002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-ea363ef0-04ab-40ef-b698-5398418e7729 None None] MFA Rules not processed for user `1131cd25fcc644c284ab11fbce35e5d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:20.451243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ea363ef0-04ab-40ef-b698-5398418e7729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:20.553469 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1105/2205] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:20.562226 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7adc2-a41e-45bf-9f20-50a74933f3af admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:20.563983 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-9dc7adc2-a41e-45bf-9f20-50a74933f3af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:20.650756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7adc2-a41e-45bf-9f20-50a74933f3af admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:20.658987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9148ea4-8f22-47dd-9f7c-cff3ff6c50a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:20.660283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9148ea4-8f22-47dd-9f7c-cff3ff6c50a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:20.660888 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9148ea4-8f22-47dd-9f7c-cff3ff6c50a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:20.661199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9148ea4-8f22-47dd-9f7c-cff3ff6c50a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:20.675901 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9148ea4-8f22-47dd-9f7c-cff3ff6c50a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:20.679403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9148ea4-8f22-47dd-9f7c-cff3ff6c50a9 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:20.696932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1101/2206] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:09:21.166626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea363ef0-04ab-40ef-b698-5398418e7729 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:21.259335 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:21.266342 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea363ef0-04ab-40ef-b698-5398418e7729 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:21.273664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a6a14-6317-4473-960f-1384a072a855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:21.275754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a6a14-6317-4473-960f-1384a072a855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:21.276138 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a6a14-6317-4473-960f-1384a072a855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:21.276505 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a6a14-6317-4473-960f-1384a072a855 admin admin] PATH_INFO: `/v3/users/3047e947bd694148955b4571df2cec62` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:21.277807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a6a14-6317-4473-960f-1384a072a855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3047e947bd694148955b4571df2cec62)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:21.279882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a6a14-6317-4473-960f-1384a072a855 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:21.352555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cb2a6a14-6317-4473-960f-1384a072a855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3047e947bd694148955b4571df2cec62'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:21.366468 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cb2a6a14-6317-4473-960f-1384a072a855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3047e947bd694148955b4571df2cec62'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:21.367677 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cb2a6a14-6317-4473-960f-1384a072a855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3047e947bd694148955b4571df2cec62'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:21.368935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cb2a6a14-6317-4473-960f-1384a072a855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3047e947bd694148955b4571df2cec62'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:21.379233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cb2a6a14-6317-4473-960f-1384a072a855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3047e947bd694148955b4571df2cec62'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:21.403999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1106/2207] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:21 2022] DELETE /identity/v3/users/3047e947bd694148955b4571df2cec62 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:21.408647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ba328-ad8d-461b-871c-7a1a40193723 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:21.409209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ba328-ad8d-461b-871c-7a1a40193723 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:21.409209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ba328-ad8d-461b-871c-7a1a40193723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:21.433849 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-d77ba328-ad8d-461b-871c-7a1a40193723 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:21.434998 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d77ba328-ad8d-461b-871c-7a1a40193723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:21.446706 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:21.518184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1102/2208] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:21.527390 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a6a14-6317-4473-960f-1384a072a855 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:21.528684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-cb2a6a14-6317-4473-960f-1384a072a855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:21.626793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a6a14-6317-4473-960f-1384a072a855 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:21.635517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e95fd7-e35b-4525-9f03-ee1bbb98d46b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:21.636258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e95fd7-e35b-4525-9f03-ee1bbb98d46b service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:21.636534 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e95fd7-e35b-4525-9f03-ee1bbb98d46b service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:21.636865 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e95fd7-e35b-4525-9f03-ee1bbb98d46b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:21.661890 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e95fd7-e35b-4525-9f03-ee1bbb98d46b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:21.666443 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e95fd7-e35b-4525-9f03-ee1bbb98d46b service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:21.683358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1107/2209] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:09:22.140432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ba328-ad8d-461b-871c-7a1a40193723 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:22.145216 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d77ba328-ad8d-461b-871c-7a1a40193723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:22.190287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ba328-ad8d-461b-871c-7a1a40193723 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:22.206465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60c9387-a9fc-46b6-b33c-d15a76feb863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:22.207413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c9387-a9fc-46b6-b33c-d15a76feb863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:22.207811 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c9387-a9fc-46b6-b33c-d15a76feb863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:22.208183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c9387-a9fc-46b6-b33c-d15a76feb863 admin admin] PATH_INFO: `/v3/projects/997959fa277e47fc84f726fd9ad2ca94` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:22.210222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c9387-a9fc-46b6-b33c-d15a76feb863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=997959fa277e47fc84f726fd9ad2ca94)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:22.221490 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60c9387-a9fc-46b6-b33c-d15a76feb863 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:22.326541 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e60c9387-a9fc-46b6-b33c-d15a76feb863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '997959fa277e47fc84f726fd9ad2ca94', 'request_id': 'req-e60c9387-a9fc-46b6-b33c-d15a76feb863'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:22.338064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e60c9387-a9fc-46b6-b33c-d15a76feb863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '997959fa277e47fc84f726fd9ad2ca94', 'request_id': 'req-e60c9387-a9fc-46b6-b33c-d15a76feb863'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:22.348762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e60c9387-a9fc-46b6-b33c-d15a76feb863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '997959fa277e47fc84f726fd9ad2ca94', 'request_id': 'req-e60c9387-a9fc-46b6-b33c-d15a76feb863'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:22.370635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e60c9387-a9fc-46b6-b33c-d15a76feb863 admin admin] The token cache is being invalidate because project 997959fa277e47fc84f726fd9ad2ca94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:22.370929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e60c9387-a9fc-46b6-b33c-d15a76feb863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:22.371635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e60c9387-a9fc-46b6-b33c-d15a76feb863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:22.379178 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1103/2210] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:22 2022] DELETE /identity/v3/projects/997959fa277e47fc84f726fd9ad2ca94 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:22.379638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e95fd7-e35b-4525-9f03-ee1bbb98d46b service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:22.390640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e95fd7-e35b-4525-9f03-ee1bbb98d46b service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:22.413574 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc787a-fac7-4a90-98df-21cdbb751695 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:22.414091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc787a-fac7-4a90-98df-21cdbb751695 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:22.414184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cc787a-fac7-4a90-98df-21cdbb751695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:22.455760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-40cc787a-fac7-4a90-98df-21cdbb751695 None None] MFA Rules not processed for user `965cbbdb06c248698ef54527c29767df`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:22.458037 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-40cc787a-fac7-4a90-98df-21cdbb751695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:22.485146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d900b31-e001-4cae-a0d2-edab89dbb681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:22.486517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d900b31-e001-4cae-a0d2-edab89dbb681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:22.486807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d900b31-e001-4cae-a0d2-edab89dbb681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:22.487202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d900b31-e001-4cae-a0d2-edab89dbb681 admin admin] PATH_INFO: `/v3/users/1131cd25fcc644c284ab11fbce35e5d6` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:22.489241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d900b31-e001-4cae-a0d2-edab89dbb681 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1131cd25fcc644c284ab11fbce35e5d6)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:22.498229 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d900b31-e001-4cae-a0d2-edab89dbb681 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:22.530043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:22.561035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1104/2211] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:22.566268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8d900b31-e001-4cae-a0d2-edab89dbb681 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1131cd25fcc644c284ab11fbce35e5d6'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:22.575368 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc787a-fac7-4a90-98df-21cdbb751695 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:22.577008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-40cc787a-fac7-4a90-98df-21cdbb751695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:22.578233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8d900b31-e001-4cae-a0d2-edab89dbb681 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1131cd25fcc644c284ab11fbce35e5d6'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:22.578939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8d900b31-e001-4cae-a0d2-edab89dbb681 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1131cd25fcc644c284ab11fbce35e5d6'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:22.579806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8d900b31-e001-4cae-a0d2-edab89dbb681 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1131cd25fcc644c284ab11fbce35e5d6'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:22.588502 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8d900b31-e001-4cae-a0d2-edab89dbb681 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1131cd25fcc644c284ab11fbce35e5d6'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:22.606568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1108/2212] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:22 2022] DELETE /identity/v3/users/1131cd25fcc644c284ab11fbce35e5d6 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:09:22.610938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:22.671000 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cc787a-fac7-4a90-98df-21cdbb751695 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:22.772998 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bc6d0-6256-4b77-9fb4-6470bba69d05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:22.773634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bc6d0-6256-4b77-9fb4-6470bba69d05 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:22.773899 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bc6d0-6256-4b77-9fb4-6470bba69d05 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:22.774309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bc6d0-6256-4b77-9fb4-6470bba69d05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:22.776522 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c81bc6d0-6256-4b77-9fb4-6470bba69d05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:22.850840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81bc6d0-6256-4b77-9fb4-6470bba69d05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:22.853827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81bc6d0-6256-4b77-9fb4-6470bba69d05 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:22.865612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1105/2213] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:09:22.998489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d900b31-e001-4cae-a0d2-edab89dbb681 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:23.000811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-8d900b31-e001-4cae-a0d2-edab89dbb681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:23.115142 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d900b31-e001-4cae-a0d2-edab89dbb681 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:23.121026 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42b8449-a8b0-4642-8a9b-c953613f7bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:23.121992 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b8449-a8b0-4642-8a9b-c953613f7bc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:23.122322 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b8449-a8b0-4642-8a9b-c953613f7bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:23.122648 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42b8449-a8b0-4642-8a9b-c953613f7bc1 admin admin] PATH_INFO: `/v3/projects/fe191b8f63b1480d85416fc57c500c2d` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:23.128272 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b8449-a8b0-4642-8a9b-c953613f7bc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe191b8f63b1480d85416fc57c500c2d)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:23.130325 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42b8449-a8b0-4642-8a9b-c953613f7bc1 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:23.207986 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e42b8449-a8b0-4642-8a9b-c953613f7bc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe191b8f63b1480d85416fc57c500c2d', 'request_id': 'req-e42b8449-a8b0-4642-8a9b-c953613f7bc1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:23.222039 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e42b8449-a8b0-4642-8a9b-c953613f7bc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe191b8f63b1480d85416fc57c500c2d', 'request_id': 'req-e42b8449-a8b0-4642-8a9b-c953613f7bc1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:23.232576 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e42b8449-a8b0-4642-8a9b-c953613f7bc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe191b8f63b1480d85416fc57c500c2d', 'request_id': 'req-e42b8449-a8b0-4642-8a9b-c953613f7bc1'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:23.301986 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e42b8449-a8b0-4642-8a9b-c953613f7bc1 admin admin] The token cache is being invalidate because project fe191b8f63b1480d85416fc57c500c2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:23.302379 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e42b8449-a8b0-4642-8a9b-c953613f7bc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:23.303255 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e42b8449-a8b0-4642-8a9b-c953613f7bc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:23.305116 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1109/2214] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:22 2022] DELETE /identity/v3/projects/fe191b8f63b1480d85416fc57c500c2d => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:23.310255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f43d8-dfa1-4d23-b9d5-4ca6619048cd None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:23.310616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f43d8-dfa1-4d23-b9d5-4ca6619048cd None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:23.310962 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f43d8-dfa1-4d23-b9d5-4ca6619048cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:23.353850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-544f43d8-dfa1-4d23-b9d5-4ca6619048cd None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:23.355248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-544f43d8-dfa1-4d23-b9d5-4ca6619048cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:23.423284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1106/2215] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:23.431388 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f7d01-e74e-4916-bcf1-f5ea2a1b3378 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:23.431926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f7d01-e74e-4916-bcf1-f5ea2a1b3378 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:23.431926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f7d01-e74e-4916-bcf1-f5ea2a1b3378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:23.455466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-bf4f7d01-e74e-4916-bcf1-f5ea2a1b3378 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:23.456775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-bf4f7d01-e74e-4916-bcf1-f5ea2a1b3378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:23.465510 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1110/2216] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:23.469944 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f43d8-dfa1-4d23-b9d5-4ca6619048cd None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:23.477280 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:23.481785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f43d8-dfa1-4d23-b9d5-4ca6619048cd None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:23.487570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9e182-e076-4ff7-8438-7993ec532480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:23.488373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9e182-e076-4ff7-8438-7993ec532480 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:23.488650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9e182-e076-4ff7-8438-7993ec532480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:23.489055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9e182-e076-4ff7-8438-7993ec532480 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:23.489720 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9e182-e076-4ff7-8438-7993ec532480 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:09:23.490042 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9e182-e076-4ff7-8438-7993ec532480 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:23.491845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9e182-e076-4ff7-8438-7993ec532480 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:23.505467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1107/2217] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:09:23.509323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f7d01-e74e-4916-bcf1-f5ea2a1b3378 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:23.527120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f7d01-e74e-4916-bcf1-f5ea2a1b3378 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:23.532386 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dac2a-a2d3-4318-8f0a-e60f939a5d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:23.533023 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dac2a-a2d3-4318-8f0a-e60f939a5d5e admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:23.533300 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dac2a-a2d3-4318-8f0a-e60f939a5d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:23.533573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dac2a-a2d3-4318-8f0a-e60f939a5d5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:23.534222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4dac2a-a2d3-4318-8f0a-e60f939a5d5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:23.536329 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4dac2a-a2d3-4318-8f0a-e60f939a5d5e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:23.565809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:23.575333 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1111/2218] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:09:23 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:23.579643 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9e182-e076-4ff7-8438-7993ec532480 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:23.592750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9e182-e076-4ff7-8438-7993ec532480 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:23.680469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab2f19-d8fc-46af-9e4d-8a59a218e657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:23.681734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab2f19-d8fc-46af-9e4d-8a59a218e657 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:23.682808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab2f19-d8fc-46af-9e4d-8a59a218e657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:23.683166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab2f19-d8fc-46af-9e4d-8a59a218e657 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:23.684800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab2f19-d8fc-46af-9e4d-8a59a218e657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:23.686741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab2f19-d8fc-46af-9e4d-8a59a218e657 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:23.715336 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1108/2219] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:09:23 2022] POST /identity/v3/users => generated 289 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:23.719794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dac2a-a2d3-4318-8f0a-e60f939a5d5e admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:23.744573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dac2a-a2d3-4318-8f0a-e60f939a5d5e admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:23.752427 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17dba6-f5bd-468a-87b5-d50fcca180a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:23.756197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17dba6-f5bd-468a-87b5-d50fcca180a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:23.756943 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17dba6-f5bd-468a-87b5-d50fcca180a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:23.757415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17dba6-f5bd-468a-87b5-d50fcca180a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:23.759315 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17dba6-f5bd-468a-87b5-d50fcca180a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:23.761570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17dba6-f5bd-468a-87b5-d50fcca180a7 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:23.787943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab2f19-d8fc-46af-9e4d-8a59a218e657 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:23.788464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1112/2220] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:23.799946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab2f19-d8fc-46af-9e4d-8a59a218e657 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:23.805907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26233652-e966-4709-b063-096bdd2f8ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:23.806708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26233652-e966-4709-b063-096bdd2f8ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:23.807052 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26233652-e966-4709-b063-096bdd2f8ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:23.807519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26233652-e966-4709-b063-096bdd2f8ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:23.808934 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26233652-e966-4709-b063-096bdd2f8ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:23.810775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26233652-e966-4709-b063-096bdd2f8ed6 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:23.827530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1109/2221] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:09:23.831951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17dba6-f5bd-468a-87b5-d50fcca180a7 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:23.844680 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17dba6-f5bd-468a-87b5-d50fcca180a7 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:23.852927 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b449b2-1fcd-4bd9-8904-a787dd387e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:23.853906 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b449b2-1fcd-4bd9-8904-a787dd387e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:23.854251 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b449b2-1fcd-4bd9-8904-a787dd387e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:23.854620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b449b2-1fcd-4bd9-8904-a787dd387e09 admin admin] PATH_INFO: `/v3/projects/adda5b2581854545a493526f6d7d7b99/users/ce2270699b3f4301bb3b7771e9aef381/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:23.872281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b449b2-1fcd-4bd9-8904-a787dd387e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adda5b2581854545a493526f6d7d7b99, user_id=ce2270699b3f4301bb3b7771e9aef381, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:23.875028 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b449b2-1fcd-4bd9-8904-a787dd387e09 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:23.893636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1113/2222] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:23 2022] PUT /identity/v3/projects/adda5b2581854545a493526f6d7d7b99/users/ce2270699b3f4301bb3b7771e9aef381/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:23.904649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879e513-d20c-4298-a507-a55dd6a4590c None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:23.905004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879e513-d20c-4298-a507-a55dd6a4590c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:23.905183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879e513-d20c-4298-a507-a55dd6a4590c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:23.940907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f879e513-d20c-4298-a507-a55dd6a4590c None None] MFA Rules not processed for user `ce2270699b3f4301bb3b7771e9aef381`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:23.942414 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f879e513-d20c-4298-a507-a55dd6a4590c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:24.072378 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1110/2223] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:24.085848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b449b2-1fcd-4bd9-8904-a787dd387e09 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:24.088306 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d3b449b2-1fcd-4bd9-8904-a787dd387e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:24.224515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b449b2-1fcd-4bd9-8904-a787dd387e09 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:24.229903 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a835b-4051-43d5-b2ae-b24cff1795df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f24ee16c043244938901c4e850dea967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:24.230534 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a835b-4051-43d5-b2ae-b24cff1795df service glance] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:24.230809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a835b-4051-43d5-b2ae-b24cff1795df service glance] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:24.231078 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a835b-4051-43d5-b2ae-b24cff1795df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:24.244160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a835b-4051-43d5-b2ae-b24cff1795df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:24.245957 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a835b-4051-43d5-b2ae-b24cff1795df service glance] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:24.256508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1114/2224] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:09:24.439867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751304f-e6b8-4650-8e7b-22d149bda2ff None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:24.440725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751304f-e6b8-4650-8e7b-22d149bda2ff None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:24.441146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751304f-e6b8-4650-8e7b-22d149bda2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:24.466374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-b751304f-e6b8-4650-8e7b-22d149bda2ff None None] MFA Rules not processed for user `965cbbdb06c248698ef54527c29767df`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:24.467697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b751304f-e6b8-4650-8e7b-22d149bda2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:24.529478 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1111/2225] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:34.252068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=50,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:34.522910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:55.963800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a835b-4051-43d5-b2ae-b24cff1795df service glance] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:56.031792 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a835b-4051-43d5-b2ae-b24cff1795df service glance] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:56.036634 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf354437-2c3f-430a-9f7c-74bf28089418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:56.037344 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf354437-2c3f-430a-9f7c-74bf28089418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:56.037676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf354437-2c3f-430a-9f7c-74bf28089418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:56.038020 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf354437-2c3f-430a-9f7c-74bf28089418 admin admin] PATH_INFO: `/v3/users/ce2270699b3f4301bb3b7771e9aef381` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:56.039336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf354437-2c3f-430a-9f7c-74bf28089418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce2270699b3f4301bb3b7771e9aef381)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:56.041348 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf354437-2c3f-430a-9f7c-74bf28089418 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:56.091765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bf354437-2c3f-430a-9f7c-74bf28089418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce2270699b3f4301bb3b7771e9aef381'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:56.100692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bf354437-2c3f-430a-9f7c-74bf28089418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce2270699b3f4301bb3b7771e9aef381'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:56.101312 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bf354437-2c3f-430a-9f7c-74bf28089418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce2270699b3f4301bb3b7771e9aef381'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:56.101926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bf354437-2c3f-430a-9f7c-74bf28089418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce2270699b3f4301bb3b7771e9aef381'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:56.109074 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bf354437-2c3f-430a-9f7c-74bf28089418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce2270699b3f4301bb3b7771e9aef381'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:56.128381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1115/2226] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:55 2022] DELETE /identity/v3/users/ce2270699b3f4301bb3b7771e9aef381 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:56.133256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e14a18-cb6d-448f-a03c-555002ebcf43 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:56.133758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e14a18-cb6d-448f-a03c-555002ebcf43 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:56.133851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e14a18-cb6d-448f-a03c-555002ebcf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:56.155198 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-c8e14a18-cb6d-448f-a03c-555002ebcf43 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:56.156636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c8e14a18-cb6d-448f-a03c-555002ebcf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:56.242970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1112/2227] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:56.253079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf354437-2c3f-430a-9f7c-74bf28089418 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:56.256403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-bf354437-2c3f-430a-9f7c-74bf28089418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:56.325528 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf354437-2c3f-430a-9f7c-74bf28089418 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:56.329446 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20860600-b5f3-4ac4-96c9-fd3c03e6b811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:56.330062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20860600-b5f3-4ac4-96c9-fd3c03e6b811 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:56.330316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20860600-b5f3-4ac4-96c9-fd3c03e6b811 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:56.330585 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20860600-b5f3-4ac4-96c9-fd3c03e6b811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:56.340405 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20860600-b5f3-4ac4-96c9-fd3c03e6b811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:56.341854 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20860600-b5f3-4ac4-96c9-fd3c03e6b811 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:56.351959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1116/2228] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:09:56.637453 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e14a18-cb6d-448f-a03c-555002ebcf43 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:56.638999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c8e14a18-cb6d-448f-a03c-555002ebcf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:56.651395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e14a18-cb6d-448f-a03c-555002ebcf43 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:56.657553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392d272-5a8a-4740-994e-286c36a4df0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:56.658133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392d272-5a8a-4740-994e-286c36a4df0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:56.658466 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392d272-5a8a-4740-994e-286c36a4df0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:56.658773 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392d272-5a8a-4740-994e-286c36a4df0d admin admin] PATH_INFO: `/v3/projects/adda5b2581854545a493526f6d7d7b99` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:56.660088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392d272-5a8a-4740-994e-286c36a4df0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=adda5b2581854545a493526f6d7d7b99)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:56.661865 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392d272-5a8a-4740-994e-286c36a4df0d admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:56.714482 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8392d272-5a8a-4740-994e-286c36a4df0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adda5b2581854545a493526f6d7d7b99', 'request_id': 'req-8392d272-5a8a-4740-994e-286c36a4df0d'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:56.722943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8392d272-5a8a-4740-994e-286c36a4df0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adda5b2581854545a493526f6d7d7b99', 'request_id': 'req-8392d272-5a8a-4740-994e-286c36a4df0d'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:56.728664 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8392d272-5a8a-4740-994e-286c36a4df0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adda5b2581854545a493526f6d7d7b99', 'request_id': 'req-8392d272-5a8a-4740-994e-286c36a4df0d'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:56.739592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8392d272-5a8a-4740-994e-286c36a4df0d admin admin] The token cache is being invalidate because project adda5b2581854545a493526f6d7d7b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:56.739875 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8392d272-5a8a-4740-994e-286c36a4df0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:56.740485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8392d272-5a8a-4740-994e-286c36a4df0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:56.741936 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1113/2229] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:56 2022] DELETE /identity/v3/projects/adda5b2581854545a493526f6d7d7b99 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:56.746950 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee0e51-222f-472e-90e4-d1edcbebc62a None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:56.747229 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee0e51-222f-472e-90e4-d1edcbebc62a None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:56.747528 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee0e51-222f-472e-90e4-d1edcbebc62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:56.767313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-a8ee0e51-222f-472e-90e4-d1edcbebc62a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:56.768558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a8ee0e51-222f-472e-90e4-d1edcbebc62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:56.785902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:56.832058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1117/2230] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:56.899402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9cb98-b639-4529-a384-ca5c8ca7cd54 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:56.899788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9cb98-b639-4529-a384-ca5c8ca7cd54 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:56.900172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd9cb98-b639-4529-a384-ca5c8ca7cd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:56.928166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-9dd9cb98-b639-4529-a384-ca5c8ca7cd54 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:56.929393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-9dd9cb98-b639-4529-a384-ca5c8ca7cd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:56.938793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1114/2231] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:56.943577 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee0e51-222f-472e-90e4-d1edcbebc62a None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:56.954329 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ee0e51-222f-472e-90e4-d1edcbebc62a None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:56.960242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dcb78f-1334-453d-8b00-bedba48e4474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:56.960882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dcb78f-1334-453d-8b00-bedba48e4474 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:56.961159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dcb78f-1334-453d-8b00-bedba48e4474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:56.961392 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dcb78f-1334-453d-8b00-bedba48e4474 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:56.962076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dcb78f-1334-453d-8b00-bedba48e4474 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:09:56.962380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dcb78f-1334-453d-8b00-bedba48e4474 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:56.964322 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dcb78f-1334-453d-8b00-bedba48e4474 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:56.964878 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9cb98-b639-4529-a384-ca5c8ca7cd54 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:56.966446 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-9dd9cb98-b639-4529-a384-ca5c8ca7cd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:56.977019 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1118/2232] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:09:56.978067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd9cb98-b639-4529-a384-ca5c8ca7cd54 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:56.981359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dcb78f-1334-453d-8b00-bedba48e4474 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:56.983378 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e851cc-e7f4-42fe-9dcc-6ef6070667c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:56.984142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e851cc-e7f4-42fe-9dcc-6ef6070667c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:56.984418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e851cc-e7f4-42fe-9dcc-6ef6070667c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:56.984683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e851cc-e7f4-42fe-9dcc-6ef6070667c2 admin admin] PATH_INFO: `/v3/users/2776b06e421d4d388c4d987596770f76` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:56.985996 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e851cc-e7f4-42fe-9dcc-6ef6070667c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2776b06e421d4d388c4d987596770f76)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:56.987951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e851cc-e7f4-42fe-9dcc-6ef6070667c2 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:56.991159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dcb78f-1334-453d-8b00-bedba48e4474 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:56.995943 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b492e-1348-42b4-93ee-42d536071ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:56.996534 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b492e-1348-42b4-93ee-42d536071ba3 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:56.996812 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b492e-1348-42b4-93ee-42d536071ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:56.997086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b492e-1348-42b4-93ee-42d536071ba3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:56.997708 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b492e-1348-42b4-93ee-42d536071ba3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:56.999639 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b492e-1348-42b4-93ee-42d536071ba3 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:57.030735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1119/2233] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:09:56 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:09:57.035065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b492e-1348-42b4-93ee-42d536071ba3 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:57.039782 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a2e851cc-e7f4-42fe-9dcc-6ef6070667c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2776b06e421d4d388c4d987596770f76'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:57.049049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a2e851cc-e7f4-42fe-9dcc-6ef6070667c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2776b06e421d4d388c4d987596770f76'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:57.049706 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a2e851cc-e7f4-42fe-9dcc-6ef6070667c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2776b06e421d4d388c4d987596770f76'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:57.050490 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a2e851cc-e7f4-42fe-9dcc-6ef6070667c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2776b06e421d4d388c4d987596770f76'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:57.064405 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a2e851cc-e7f4-42fe-9dcc-6ef6070667c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2776b06e421d4d388c4d987596770f76'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:57.065605 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:57.079882 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1115/2234] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:09:56 2022] DELETE /identity/v3/users/2776b06e421d4d388c4d987596770f76 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:57.084341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d136ab-52e4-4230-88ad-47425ce5b3a6 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:57.084565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d136ab-52e4-4230-88ad-47425ce5b3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:57.084863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d136ab-52e4-4230-88ad-47425ce5b3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:57.099245 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b492e-1348-42b4-93ee-42d536071ba3 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:57.103771 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-69d136ab-52e4-4230-88ad-47425ce5b3a6 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:57.105048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-69d136ab-52e4-4230-88ad-47425ce5b3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:57.162155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e91215-06f5-41a0-a565-2a13e6a80b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:57.162791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e91215-06f5-41a0-a565-2a13e6a80b73 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:57.163064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e91215-06f5-41a0-a565-2a13e6a80b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:57.163331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e91215-06f5-41a0-a565-2a13e6a80b73 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:57.163975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e91215-06f5-41a0-a565-2a13e6a80b73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:57.164184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1116/2235] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:57.171071 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e91215-06f5-41a0-a565-2a13e6a80b73 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:57.174255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d136ab-52e4-4230-88ad-47425ce5b3a6 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:57.175671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-69d136ab-52e4-4230-88ad-47425ce5b3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:57.188924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1120/2236] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:09:57 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:09:57.192605 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e91215-06f5-41a0-a565-2a13e6a80b73 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:57.194594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f2e91215-06f5-41a0-a565-2a13e6a80b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:57.206653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e91215-06f5-41a0-a565-2a13e6a80b73 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:57.212074 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64031ca-23e0-4a19-b144-0a754febb122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:57.212717 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64031ca-23e0-4a19-b144-0a754febb122 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:57.213020 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64031ca-23e0-4a19-b144-0a754febb122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:57.213288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64031ca-23e0-4a19-b144-0a754febb122 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:57.213939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64031ca-23e0-4a19-b144-0a754febb122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:57.215626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64031ca-23e0-4a19-b144-0a754febb122 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:57.226014 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1121/2237] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:09:57.229946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64031ca-23e0-4a19-b144-0a754febb122 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:57.235326 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d136ab-52e4-4230-88ad-47425ce5b3a6 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:57.238947 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4b072-4bd6-4875-b9b8-a3fe2c66b532 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:57.239549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64031ca-23e0-4a19-b144-0a754febb122 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:57.239704 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4b072-4bd6-4875-b9b8-a3fe2c66b532 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:57.239852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4b072-4bd6-4875-b9b8-a3fe2c66b532 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:57.240116 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4b072-4bd6-4875-b9b8-a3fe2c66b532 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:57.244065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31737c2a-a2cb-4c81-a7a5-8f8d6d16c380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:57.244658 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31737c2a-a2cb-4c81-a7a5-8f8d6d16c380 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:57.244934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31737c2a-a2cb-4c81-a7a5-8f8d6d16c380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:57.245202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31737c2a-a2cb-4c81-a7a5-8f8d6d16c380 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:57.245842 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31737c2a-a2cb-4c81-a7a5-8f8d6d16c380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:57.247596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31737c2a-a2cb-4c81-a7a5-8f8d6d16c380 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:57.249866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4b072-4bd6-4875-b9b8-a3fe2c66b532 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:57.251295 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4b072-4bd6-4875-b9b8-a3fe2c66b532 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:57.258810 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1122/2238] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:09:57.261985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1117/2239] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:09:57.264302 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31737c2a-a2cb-4c81-a7a5-8f8d6d16c380 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:57.269113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:57.274631 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31737c2a-a2cb-4c81-a7a5-8f8d6d16c380 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:57.280799 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3091e6-ca70-4057-8543-9d26d5a2d43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:57.281452 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3091e6-ca70-4057-8543-9d26d5a2d43b admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:57.281722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3091e6-ca70-4057-8543-9d26d5a2d43b admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:57.281987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3091e6-ca70-4057-8543-9d26d5a2d43b admin admin] PATH_INFO: `/v3/projects/19a3e27d765147298f6d421fc614f77f/users/f66138a7d2594933b09e1fcc7a4c2f1a/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:57.298875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3091e6-ca70-4057-8543-9d26d5a2d43b admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a3e27d765147298f6d421fc614f77f, user_id=f66138a7d2594933b09e1fcc7a4c2f1a, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:57.301646 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3091e6-ca70-4057-8543-9d26d5a2d43b admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:57.317324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1123/2240] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:57 2022] PUT /identity/v3/projects/19a3e27d765147298f6d421fc614f77f/users/f66138a7d2594933b09e1fcc7a4c2f1a/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:09:57.328301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55fe9a-e14d-4780-8bff-47a7fdd2b1f3 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:57.328583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55fe9a-e14d-4780-8bff-47a7fdd2b1f3 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:57.328855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55fe9a-e14d-4780-8bff-47a7fdd2b1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:57.357374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-5d55fe9a-e14d-4780-8bff-47a7fdd2b1f3 None None] MFA Rules not processed for user `f66138a7d2594933b09e1fcc7a4c2f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:57.358712 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5d55fe9a-e14d-4780-8bff-47a7fdd2b1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:57.438198 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1118/2241] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:57.449475 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3091e6-ca70-4057-8543-9d26d5a2d43b admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:57.451463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-eb3091e6-ca70-4057-8543-9d26d5a2d43b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:57.510403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3091e6-ca70-4057-8543-9d26d5a2d43b admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:57.514395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa7cd3-9b50-4528-8b46-9f3ee76dd557 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:57.515058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa7cd3-9b50-4528-8b46-9f3ee76dd557 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:57.515335 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa7cd3-9b50-4528-8b46-9f3ee76dd557 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:57.515618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa7cd3-9b50-4528-8b46-9f3ee76dd557 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:57.524981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa7cd3-9b50-4528-8b46-9f3ee76dd557 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:57.526487 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa7cd3-9b50-4528-8b46-9f3ee76dd557 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:57.535771 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1124/2242] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:09:57.630655 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55fe9a-e14d-4780-8bff-47a7fdd2b1f3 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:57.634030 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5d55fe9a-e14d-4780-8bff-47a7fdd2b1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:57.678928 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:57.725299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55fe9a-e14d-4780-8bff-47a7fdd2b1f3 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:57.730389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7129d94-ccca-4fd5-a1a3-4ea6ea8adab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:57.731158 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7129d94-ccca-4fd5-a1a3-4ea6ea8adab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:57.731660 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7129d94-ccca-4fd5-a1a3-4ea6ea8adab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:57.732057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7129d94-ccca-4fd5-a1a3-4ea6ea8adab7 admin admin] PATH_INFO: `/v3/projects/4f49764f81904d7faa8a203eef70ec24` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:57.733186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7129d94-ccca-4fd5-a1a3-4ea6ea8adab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f49764f81904d7faa8a203eef70ec24)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:57.735112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7129d94-ccca-4fd5-a1a3-4ea6ea8adab7 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:57.792669 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d7129d94-ccca-4fd5-a1a3-4ea6ea8adab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f49764f81904d7faa8a203eef70ec24', 'request_id': 'req-d7129d94-ccca-4fd5-a1a3-4ea6ea8adab7'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:57.811232 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d7129d94-ccca-4fd5-a1a3-4ea6ea8adab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f49764f81904d7faa8a203eef70ec24', 'request_id': 'req-d7129d94-ccca-4fd5-a1a3-4ea6ea8adab7'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:57.818650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d7129d94-ccca-4fd5-a1a3-4ea6ea8adab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f49764f81904d7faa8a203eef70ec24', 'request_id': 'req-d7129d94-ccca-4fd5-a1a3-4ea6ea8adab7'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:57.841982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d7129d94-ccca-4fd5-a1a3-4ea6ea8adab7 admin admin] The token cache is being invalidate because project 4f49764f81904d7faa8a203eef70ec24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:09:57.842325 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d7129d94-ccca-4fd5-a1a3-4ea6ea8adab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:57.842959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d7129d94-ccca-4fd5-a1a3-4ea6ea8adab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:09:57.844815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1119/2243] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:09:57 2022] DELETE /identity/v3/projects/4f49764f81904d7faa8a203eef70ec24 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:57.851220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a36e9-7fdb-45c5-bd4d-b01482f62ecd None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:57.851543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a36e9-7fdb-45c5-bd4d-b01482f62ecd None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:57.851820 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a36e9-7fdb-45c5-bd4d-b01482f62ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:57.877268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-f73a36e9-7fdb-45c5-bd4d-b01482f62ecd None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:57.878604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f73a36e9-7fdb-45c5-bd4d-b01482f62ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:57.937089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1125/2244] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:57.945413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03fcfe-04c7-47ab-8b14-66d75a04c8ad None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:57.945832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03fcfe-04c7-47ab-8b14-66d75a04c8ad None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:57.945991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03fcfe-04c7-47ab-8b14-66d75a04c8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:57.966889 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-8d03fcfe-04c7-47ab-8b14-66d75a04c8ad None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:57.968111 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8d03fcfe-04c7-47ab-8b14-66d75a04c8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:57.975977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1120/2245] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:57.980413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a36e9-7fdb-45c5-bd4d-b01482f62ecd None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:57.990727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a36e9-7fdb-45c5-bd4d-b01482f62ecd None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:57.996240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548143c9-6d67-46a3-b7ff-96d68d7c8344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:57.996936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548143c9-6d67-46a3-b7ff-96d68d7c8344 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:57.997244 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548143c9-6d67-46a3-b7ff-96d68d7c8344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:57.997513 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548143c9-6d67-46a3-b7ff-96d68d7c8344 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:57.998205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548143c9-6d67-46a3-b7ff-96d68d7c8344 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:09:57.998493 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548143c9-6d67-46a3-b7ff-96d68d7c8344 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:58.000152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548143c9-6d67-46a3-b7ff-96d68d7c8344 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:58.012630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1126/2246] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:09:58.016860 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03fcfe-04c7-47ab-8b14-66d75a04c8ad None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:58.025844 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03fcfe-04c7-47ab-8b14-66d75a04c8ad None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:58.030462 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e242c18d-92f7-4d98-b7d7-29cfd3d72ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:58.031053 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242c18d-92f7-4d98-b7d7-29cfd3d72ce5 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:58.031325 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242c18d-92f7-4d98-b7d7-29cfd3d72ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:58.031632 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e242c18d-92f7-4d98-b7d7-29cfd3d72ce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:58.032280 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e242c18d-92f7-4d98-b7d7-29cfd3d72ce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:58.034166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e242c18d-92f7-4d98-b7d7-29cfd3d72ce5 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:58.063493 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1121/2247] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:09:58 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:09:58.067970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548143c9-6d67-46a3-b7ff-96d68d7c8344 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:58.078724 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548143c9-6d67-46a3-b7ff-96d68d7c8344 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:58.140737 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c1a84-e93f-4c5b-b45e-7c85dedef593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:58.141269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c1a84-e93f-4c5b-b45e-7c85dedef593 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:58.141570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c1a84-e93f-4c5b-b45e-7c85dedef593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:58.141835 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c1a84-e93f-4c5b-b45e-7c85dedef593 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:58.142506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c1a84-e93f-4c5b-b45e-7c85dedef593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:58.144511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c1a84-e93f-4c5b-b45e-7c85dedef593 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:58.162849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:58.169545 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1127/2248] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:09:58 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:09:58.174177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e242c18d-92f7-4d98-b7d7-29cfd3d72ce5 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:58.186525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e242c18d-92f7-4d98-b7d7-29cfd3d72ce5 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:58.191306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51795495-4c61-4483-ab6f-19ff42cae6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:58.192055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51795495-4c61-4483-ab6f-19ff42cae6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:58.192329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51795495-4c61-4483-ab6f-19ff42cae6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:58.192645 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51795495-4c61-4483-ab6f-19ff42cae6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:58.193285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51795495-4c61-4483-ab6f-19ff42cae6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:58.195260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51795495-4c61-4483-ab6f-19ff42cae6a1 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:58.205665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1122/2249] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:58.210360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c1a84-e93f-4c5b-b45e-7c85dedef593 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:58.220876 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c1a84-e93f-4c5b-b45e-7c85dedef593 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:58.226725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c4205-ce98-4fb4-b96e-8a393d7749f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:58.227488 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c4205-ce98-4fb4-b96e-8a393d7749f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:58.227775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c4205-ce98-4fb4-b96e-8a393d7749f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:58.228101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c4205-ce98-4fb4-b96e-8a393d7749f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:58.228829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c4205-ce98-4fb4-b96e-8a393d7749f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:58.230625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c4205-ce98-4fb4-b96e-8a393d7749f1 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:58.243510 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1128/2250] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:09:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:09:58.248097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51795495-4c61-4483-ab6f-19ff42cae6a1 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:58.258693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51795495-4c61-4483-ab6f-19ff42cae6a1 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:58.263396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4396e49-5638-4fc4-90a2-165ef35c7e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:58.264094 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4396e49-5638-4fc4-90a2-165ef35c7e12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:58.264370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4396e49-5638-4fc4-90a2-165ef35c7e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:58.264641 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4396e49-5638-4fc4-90a2-165ef35c7e12 admin admin] PATH_INFO: `/v3/projects/a942a68e1c5041fca911ac30322ee0d9/users/118b7b52617c4679b4b5da8a00cbff7e/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:58.282369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4396e49-5638-4fc4-90a2-165ef35c7e12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a942a68e1c5041fca911ac30322ee0d9, user_id=118b7b52617c4679b4b5da8a00cbff7e, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:58.285517 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4396e49-5638-4fc4-90a2-165ef35c7e12 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:58.300559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1123/2251] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:09:58 2022] PUT /identity/v3/projects/a942a68e1c5041fca911ac30322ee0d9/users/118b7b52617c4679b4b5da8a00cbff7e/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:09:58.311098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf268a2a-21ae-4898-bcd6-9e301c5b2b9d None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:58.311382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf268a2a-21ae-4898-bcd6-9e301c5b2b9d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:58.311687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf268a2a-21ae-4898-bcd6-9e301c5b2b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:58.344210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-cf268a2a-21ae-4898-bcd6-9e301c5b2b9d None None] MFA Rules not processed for user `118b7b52617c4679b4b5da8a00cbff7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:58.345440 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-cf268a2a-21ae-4898-bcd6-9e301c5b2b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:58.417123 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1129/2252] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:09:58.424908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4396e49-5638-4fc4-90a2-165ef35c7e12 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:09:58.426738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b4396e49-5638-4fc4-90a2-165ef35c7e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:58.480255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:09:58.484367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4396e49-5638-4fc4-90a2-165ef35c7e12 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:09:58.489712 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b974ed-ad59-4794-8791-220073a9101e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:09:58.490681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b974ed-ad59-4794-8791-220073a9101e service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:58.491170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b974ed-ad59-4794-8791-220073a9101e service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:58.491438 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b974ed-ad59-4794-8791-220073a9101e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:58.502193 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b974ed-ad59-4794-8791-220073a9101e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:09:58.505288 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b974ed-ad59-4794-8791-220073a9101e service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:09:58.516253 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1124/2253] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:09:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:09:58.955274 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2193908-da81-44a6-8d08-76d58c2179f2 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:09:58.955604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2193908-da81-44a6-8d08-76d58c2179f2 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:09:58.955977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2193908-da81-44a6-8d08-76d58c2179f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:09:58.978305 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-c2193908-da81-44a6-8d08-76d58c2179f2 None None] MFA Rules not processed for user `f66138a7d2594933b09e1fcc7a4c2f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:09:58.979670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c2193908-da81-44a6-8d08-76d58c2179f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:09:59.053769 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1130/2254] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:09:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:08.510695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:10:09.041985 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=63 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:10:10.067540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443006e3-7364-4bb6-be45-ed372bea238c None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:10.068428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443006e3-7364-4bb6-be45-ed372bea238c None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:10.069092 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443006e3-7364-4bb6-be45-ed372bea238c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:10.092662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-443006e3-7364-4bb6-be45-ed372bea238c None None] MFA Rules not processed for user `2127487c50884141b2776f05e96471fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:10.093405 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-443006e3-7364-4bb6-be45-ed372bea238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:10.167806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1125/2255] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:10:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:12.442021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2193908-da81-44a6-8d08-76d58c2179f2 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:12.511995 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2193908-da81-44a6-8d08-76d58c2179f2 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:12.516996 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7e07e-58cc-44f1-afc5-84b3f14c463c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:12.517636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7e07e-58cc-44f1-afc5-84b3f14c463c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:12.517912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7e07e-58cc-44f1-afc5-84b3f14c463c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:12.518180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7e07e-58cc-44f1-afc5-84b3f14c463c admin admin] PATH_INFO: `/v3/users/118b7b52617c4679b4b5da8a00cbff7e` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:12.519325 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7e07e-58cc-44f1-afc5-84b3f14c463c admin admin] RBAC: Authorizing `identity:delete_user(user_id=118b7b52617c4679b4b5da8a00cbff7e)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:12.521330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7e07e-58cc-44f1-afc5-84b3f14c463c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:12.572690 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1cf7e07e-58cc-44f1-afc5-84b3f14c463c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '118b7b52617c4679b4b5da8a00cbff7e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:12.580904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1cf7e07e-58cc-44f1-afc5-84b3f14c463c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '118b7b52617c4679b4b5da8a00cbff7e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:12.581539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1cf7e07e-58cc-44f1-afc5-84b3f14c463c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '118b7b52617c4679b4b5da8a00cbff7e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:12.582200 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1cf7e07e-58cc-44f1-afc5-84b3f14c463c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '118b7b52617c4679b4b5da8a00cbff7e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:12.589196 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1cf7e07e-58cc-44f1-afc5-84b3f14c463c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '118b7b52617c4679b4b5da8a00cbff7e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:12.605991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1131/2256] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:10:12 2022] DELETE /identity/v3/users/118b7b52617c4679b4b5da8a00cbff7e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:10:12.610762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2e75b-fc81-4974-abac-c7b049c3db06 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:12.611229 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2e75b-fc81-4974-abac-c7b049c3db06 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:12.611321 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2e75b-fc81-4974-abac-c7b049c3db06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:12.630873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-83e2e75b-fc81-4974-abac-c7b049c3db06 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:12.632562 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-83e2e75b-fc81-4974-abac-c7b049c3db06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:12.698215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1126/2257] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:12.708453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7e07e-58cc-44f1-afc5-84b3f14c463c admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:12.710157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1cf7e07e-58cc-44f1-afc5-84b3f14c463c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:12.780240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7e07e-58cc-44f1-afc5-84b3f14c463c admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:12.784394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe84ec-d3ea-44ef-9870-4cc741c8123e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:12.784989 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe84ec-d3ea-44ef-9870-4cc741c8123e service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:12.785263 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe84ec-d3ea-44ef-9870-4cc741c8123e service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:12.785527 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe84ec-d3ea-44ef-9870-4cc741c8123e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:12.794732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe84ec-d3ea-44ef-9870-4cc741c8123e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:12.796204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe84ec-d3ea-44ef-9870-4cc741c8123e service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:12.805694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1132/2258] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:10:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:10:13.161273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2e75b-fc81-4974-abac-c7b049c3db06 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:13.172420 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-83e2e75b-fc81-4974-abac-c7b049c3db06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:13.185597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2e75b-fc81-4974-abac-c7b049c3db06 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:13.190397 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252a39c-d955-45a4-90ca-f5a6d3103ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:13.191090 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252a39c-d955-45a4-90ca-f5a6d3103ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:13.191432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252a39c-d955-45a4-90ca-f5a6d3103ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:13.191830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252a39c-d955-45a4-90ca-f5a6d3103ca9 admin admin] PATH_INFO: `/v3/projects/a942a68e1c5041fca911ac30322ee0d9` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:13.193720 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252a39c-d955-45a4-90ca-f5a6d3103ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a942a68e1c5041fca911ac30322ee0d9)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:13.196317 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252a39c-d955-45a4-90ca-f5a6d3103ca9 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:13.268952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-2252a39c-d955-45a4-90ca-f5a6d3103ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a942a68e1c5041fca911ac30322ee0d9', 'request_id': 'req-2252a39c-d955-45a4-90ca-f5a6d3103ca9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:13.276532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-2252a39c-d955-45a4-90ca-f5a6d3103ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a942a68e1c5041fca911ac30322ee0d9', 'request_id': 'req-2252a39c-d955-45a4-90ca-f5a6d3103ca9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:13.282686 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-2252a39c-d955-45a4-90ca-f5a6d3103ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a942a68e1c5041fca911ac30322ee0d9', 'request_id': 'req-2252a39c-d955-45a4-90ca-f5a6d3103ca9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:13.293952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-2252a39c-d955-45a4-90ca-f5a6d3103ca9 admin admin] The token cache is being invalidate because project a942a68e1c5041fca911ac30322ee0d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:10:13.294238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-2252a39c-d955-45a4-90ca-f5a6d3103ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:13.294863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-2252a39c-d955-45a4-90ca-f5a6d3103ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:13.296488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1127/2259] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:10:13 2022] DELETE /identity/v3/projects/a942a68e1c5041fca911ac30322ee0d9 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:10:13.301547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7c447-69b5-43d5-a60c-6589269448a5 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:13.301832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7c447-69b5-43d5-a60c-6589269448a5 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:13.302106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7c447-69b5-43d5-a60c-6589269448a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:13.322737 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-96c7c447-69b5-43d5-a60c-6589269448a5 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:13.323991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-96c7c447-69b5-43d5-a60c-6589269448a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:13.341693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:10:13.390509 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1133/2260] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:13.398777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d4a58-2143-418a-997e-3b97f444707f None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:13.399057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d4a58-2143-418a-997e-3b97f444707f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:13.399334 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d4a58-2143-418a-997e-3b97f444707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:13.425131 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-415d4a58-2143-418a-997e-3b97f444707f None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:13.426460 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-415d4a58-2143-418a-997e-3b97f444707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:13.435517 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1128/2261] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:13.439941 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7c447-69b5-43d5-a60c-6589269448a5 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:13.452719 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7c447-69b5-43d5-a60c-6589269448a5 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:13.460191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc5e02-f4f9-4265-97eb-ee3a2958dd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:13.460831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc5e02-f4f9-4265-97eb-ee3a2958dd81 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:13.461110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc5e02-f4f9-4265-97eb-ee3a2958dd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:13.461429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc5e02-f4f9-4265-97eb-ee3a2958dd81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:13.462128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc5e02-f4f9-4265-97eb-ee3a2958dd81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:10:13.462455 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc5e02-f4f9-4265-97eb-ee3a2958dd81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:13.464166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc5e02-f4f9-4265-97eb-ee3a2958dd81 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:13.481766 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1134/2262] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:10:13.486459 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d4a58-2143-418a-997e-3b97f444707f None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:13.498743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d4a58-2143-418a-997e-3b97f444707f None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:13.511855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901638c6-5dd3-45c4-b3ef-c112fe15e323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:13.513117 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901638c6-5dd3-45c4-b3ef-c112fe15e323 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:13.513286 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901638c6-5dd3-45c4-b3ef-c112fe15e323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:13.513553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901638c6-5dd3-45c4-b3ef-c112fe15e323 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:13.514614 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901638c6-5dd3-45c4-b3ef-c112fe15e323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:13.516740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901638c6-5dd3-45c4-b3ef-c112fe15e323 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:13.548993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:10:13.558343 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1129/2263] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:10:13 2022] POST /identity/v3/projects => generated 377 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:10:13.563152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc5e02-f4f9-4265-97eb-ee3a2958dd81 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:13.574462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc5e02-f4f9-4265-97eb-ee3a2958dd81 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:13.659656 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7678fdb-6cbb-47a2-a6f4-feec44c0bbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:13.660536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7678fdb-6cbb-47a2-a6f4-feec44c0bbd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:13.661159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7678fdb-6cbb-47a2-a6f4-feec44c0bbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:13.661590 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7678fdb-6cbb-47a2-a6f4-feec44c0bbd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:13.662373 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7678fdb-6cbb-47a2-a6f4-feec44c0bbd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:13.664632 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7678fdb-6cbb-47a2-a6f4-feec44c0bbd7 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:13.686303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1135/2264] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:10:13 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:10:13.690290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901638c6-5dd3-45c4-b3ef-c112fe15e323 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:13.700786 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901638c6-5dd3-45c4-b3ef-c112fe15e323 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:13.705280 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3e846-24da-494c-aa99-f71d5e0711e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:13.705872 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3e846-24da-494c-aa99-f71d5e0711e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:13.706144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3e846-24da-494c-aa99-f71d5e0711e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:13.706408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3e846-24da-494c-aa99-f71d5e0711e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:13.707046 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3e846-24da-494c-aa99-f71d5e0711e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:13.708687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3e846-24da-494c-aa99-f71d5e0711e5 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:13.718474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1130/2265] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:10:13.722375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7678fdb-6cbb-47a2-a6f4-feec44c0bbd7 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:13.732474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7678fdb-6cbb-47a2-a6f4-feec44c0bbd7 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:13.737599 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382eb034-42af-4b5a-bfc1-0828516881ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:13.738180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382eb034-42af-4b5a-bfc1-0828516881ad admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:13.738455 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382eb034-42af-4b5a-bfc1-0828516881ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:13.738723 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382eb034-42af-4b5a-bfc1-0828516881ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:13.739363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382eb034-42af-4b5a-bfc1-0828516881ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:13.740987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382eb034-42af-4b5a-bfc1-0828516881ad admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:13.751910 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1136/2266] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:10:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:10:13.756623 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3e846-24da-494c-aa99-f71d5e0711e5 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:13.766987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3e846-24da-494c-aa99-f71d5e0711e5 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:13.772743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea3b5b-98bf-46f9-b880-2e18f79253af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:13.773466 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea3b5b-98bf-46f9-b880-2e18f79253af admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:13.773699 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea3b5b-98bf-46f9-b880-2e18f79253af admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:13.773968 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea3b5b-98bf-46f9-b880-2e18f79253af admin admin] PATH_INFO: `/v3/projects/53275b4dc845465d8b0bfb50eacd14f6/users/77aeaed3d55945e49b0eb6e3c0519294/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:13.790566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea3b5b-98bf-46f9-b880-2e18f79253af admin admin] RBAC: Authorizing `identity:create_grant(project_id=53275b4dc845465d8b0bfb50eacd14f6, user_id=77aeaed3d55945e49b0eb6e3c0519294, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:13.793299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea3b5b-98bf-46f9-b880-2e18f79253af admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:13.808125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1131/2267] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:10:13 2022] PUT /identity/v3/projects/53275b4dc845465d8b0bfb50eacd14f6/users/77aeaed3d55945e49b0eb6e3c0519294/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:10:13.813383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061daad2-2e51-4e63-9149-756d94470c67 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:13.813687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061daad2-2e51-4e63-9149-756d94470c67 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:13.813977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061daad2-2e51-4e63-9149-756d94470c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:13.833552 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-061daad2-2e51-4e63-9149-756d94470c67 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:13.834742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-061daad2-2e51-4e63-9149-756d94470c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:13.900117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1137/2268] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:13.909976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea3b5b-98bf-46f9-b880-2e18f79253af admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:13.911821 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-06ea3b5b-98bf-46f9-b880-2e18f79253af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:13.968666 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea3b5b-98bf-46f9-b880-2e18f79253af admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:13.972667 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7c7f7-dafd-422b-a8d4-d2225ab155c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:13.973266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7c7f7-dafd-422b-a8d4-d2225ab155c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:13.973534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7c7f7-dafd-422b-a8d4-d2225ab155c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:13.973783 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7c7f7-dafd-422b-a8d4-d2225ab155c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:13.982798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff7c7f7-dafd-422b-a8d4-d2225ab155c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:13.984369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff7c7f7-dafd-422b-a8d4-d2225ab155c1 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:13.994408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1132/2269] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:10:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:10:19.761715 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2ca01-ec6d-4910-b62e-f8aea9300504 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:19.762285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2ca01-ec6d-4910-b62e-f8aea9300504 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:19.762285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a2ca01-ec6d-4910-b62e-f8aea9300504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:19.797813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-b3a2ca01-ec6d-4910-b62e-f8aea9300504 None None] MFA Rules not processed for user `77aeaed3d55945e49b0eb6e3c0519294`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:19.799692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b3a2ca01-ec6d-4910-b62e-f8aea9300504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:19.824185 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:10:19.930332 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7c7f7-dafd-422b-a8d4-d2225ab155c1 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:19.931249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1138/2270] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:10:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:19.942898 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7c7f7-dafd-422b-a8d4-d2225ab155c1 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:19.950389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf5fa1-44bb-42c2-acc7-a776946700d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:19.951002 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf5fa1-44bb-42c2-acc7-a776946700d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:19.951273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf5fa1-44bb-42c2-acc7-a776946700d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:19.951567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf5fa1-44bb-42c2-acc7-a776946700d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:19.952373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf5fa1-44bb-42c2-acc7-a776946700d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:19.954556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf5fa1-44bb-42c2-acc7-a776946700d9 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:19.991121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1133/2271] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:10:19 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:10:19.994293 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf5fa1-44bb-42c2-acc7-a776946700d9 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:19.994758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5315b-ad95-48b1-80d2-f870a69dbe16 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:19.996594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5315b-ad95-48b1-80d2-f870a69dbe16 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:19.996886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e5315b-ad95-48b1-80d2-f870a69dbe16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:20.028860 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-a7e5315b-ad95-48b1-80d2-f870a69dbe16 None None] MFA Rules not processed for user `18375266a64348df879a9eec53bfcbf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:20.030161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a7e5315b-ad95-48b1-80d2-f870a69dbe16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:20.069635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf5fa1-44bb-42c2-acc7-a776946700d9 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:20.074841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cace0f-8482-481b-ba42-d6ffaa6f5c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:20.075651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cace0f-8482-481b-ba42-d6ffaa6f5c33 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:20.076027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cace0f-8482-481b-ba42-d6ffaa6f5c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:20.076528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cace0f-8482-481b-ba42-d6ffaa6f5c33 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:20.077439 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cace0f-8482-481b-ba42-d6ffaa6f5c33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:20.079930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cace0f-8482-481b-ba42-d6ffaa6f5c33 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:20.107840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1134/2272] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:10:19 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:10:20.111603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cace0f-8482-481b-ba42-d6ffaa6f5c33 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:20.112626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1139/2273] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:10:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:20.128488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cace0f-8482-481b-ba42-d6ffaa6f5c33 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:20.133431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce649c9-9f61-4d02-89db-b3e129e8b4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:20.134059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce649c9-9f61-4d02-89db-b3e129e8b4b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:20.134330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce649c9-9f61-4d02-89db-b3e129e8b4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:20.134592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce649c9-9f61-4d02-89db-b3e129e8b4b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:20.135266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce649c9-9f61-4d02-89db-b3e129e8b4b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:20.138489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce649c9-9f61-4d02-89db-b3e129e8b4b5 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:20.151264 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1135/2274] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:10:20.155344 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5315b-ad95-48b1-80d2-f870a69dbe16 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:20.204105 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e5315b-ad95-48b1-80d2-f870a69dbe16 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:20.209559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cfd28-3d98-4ad6-9356-32fee4b23f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:20.210435 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cfd28-3d98-4ad6-9356-32fee4b23f3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:20.210740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cfd28-3d98-4ad6-9356-32fee4b23f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:20.211015 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cfd28-3d98-4ad6-9356-32fee4b23f3a admin admin] PATH_INFO: `/v3/projects/19d1c7a3b9154b0aad0c4286f3199772/users/02cddad0448d45bb9458d5cc7cdfbc5a/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:20.230429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cfd28-3d98-4ad6-9356-32fee4b23f3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d1c7a3b9154b0aad0c4286f3199772, user_id=02cddad0448d45bb9458d5cc7cdfbc5a, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:20.232485 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567cfd28-3d98-4ad6-9356-32fee4b23f3a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:20.264077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1140/2275] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:10:20 2022] PUT /identity/v3/projects/19d1c7a3b9154b0aad0c4286f3199772/users/02cddad0448d45bb9458d5cc7cdfbc5a/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:10:20.265463 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce649c9-9f61-4d02-89db-b3e129e8b4b5 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:20.278214 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce649c9-9f61-4d02-89db-b3e129e8b4b5 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:20.302406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:10:20.364225 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e022228-4898-4784-973d-a2387adcf30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:20.364967 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e022228-4898-4784-973d-a2387adcf30c admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:20.365748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e022228-4898-4784-973d-a2387adcf30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:20.365748 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e022228-4898-4784-973d-a2387adcf30c admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:20.367082 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e022228-4898-4784-973d-a2387adcf30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:20.368855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e022228-4898-4784-973d-a2387adcf30c admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:20.383485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1136/2276] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:10:20.385650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cfd28-3d98-4ad6-9356-32fee4b23f3a admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:20.398177 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567cfd28-3d98-4ad6-9356-32fee4b23f3a admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:20.403430 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf679e-c2a4-4a27-8924-0455d71d03ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:20.404697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf679e-c2a4-4a27-8924-0455d71d03ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:20.404697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf679e-c2a4-4a27-8924-0455d71d03ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:20.404897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf679e-c2a4-4a27-8924-0455d71d03ca admin admin] PATH_INFO: `/v3/projects/19d1c7a3b9154b0aad0c4286f3199772/users/02cddad0448d45bb9458d5cc7cdfbc5a/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:20.407090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf679e-c2a4-4a27-8924-0455d71d03ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d1c7a3b9154b0aad0c4286f3199772, user_id=02cddad0448d45bb9458d5cc7cdfbc5a, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:20.410360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf679e-c2a4-4a27-8924-0455d71d03ca admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:20.427623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1141/2277] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:10:20 2022] PUT /identity/v3/projects/19d1c7a3b9154b0aad0c4286f3199772/users/02cddad0448d45bb9458d5cc7cdfbc5a/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:10:20.683383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc00cd-4e41-4d7c-b53f-168982faccbe None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:20.683737 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc00cd-4e41-4d7c-b53f-168982faccbe None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:20.683958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc00cd-4e41-4d7c-b53f-168982faccbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:20.729235 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-7cfc00cd-4e41-4d7c-b53f-168982faccbe None None] MFA Rules not processed for user `00b673d5c65d40d9a90cc3b579e83e7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:20.731803 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-7cfc00cd-4e41-4d7c-b53f-168982faccbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:20.833034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1137/2278] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:10:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:26.403530 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08abaf-3357-477a-9c77-e5c34ca17cb0 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:26.404108 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08abaf-3357-477a-9c77-e5c34ca17cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:26.404108 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf08abaf-3357-477a-9c77-e5c34ca17cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:26.446836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-cf08abaf-3357-477a-9c77-e5c34ca17cb0 None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:26.448474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-cf08abaf-3357-477a-9c77-e5c34ca17cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:26.572693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1142/2279] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Sep 8 16:10:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:26.626993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc00cd-4e41-4d7c-b53f-168982faccbe None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:26.629149 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-7cfc00cd-4e41-4d7c-b53f-168982faccbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:26.741946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc00cd-4e41-4d7c-b53f-168982faccbe None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:26.752159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c87fcd-8e9e-4fbd-978c-b45f1ef33aec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:26.753064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c87fcd-8e9e-4fbd-978c-b45f1ef33aec service swift] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:26.753357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c87fcd-8e9e-4fbd-978c-b45f1ef33aec service swift] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:26.753785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c87fcd-8e9e-4fbd-978c-b45f1ef33aec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:26.767141 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c87fcd-8e9e-4fbd-978c-b45f1ef33aec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:26.768821 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c87fcd-8e9e-4fbd-978c-b45f1ef33aec service swift] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:26.787840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1138/2280] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:10:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:10:27.992420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28c21b-1297-4bc3-8351-fd20ab2189dd None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:27.994459 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28c21b-1297-4bc3-8351-fd20ab2189dd None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:27.995252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28c21b-1297-4bc3-8351-fd20ab2189dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:28.063976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-bb28c21b-1297-4bc3-8351-fd20ab2189dd None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:28.067671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-bb28c21b-1297-4bc3-8351-fd20ab2189dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:28.088291 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1143/2281] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Sep 8 16:10:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:28.103416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c87fcd-8e9e-4fbd-978c-b45f1ef33aec service swift] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:28.129008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c87fcd-8e9e-4fbd-978c-b45f1ef33aec service swift] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:28.141642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b80f45-2950-486c-98f8-f95f3d6377d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:28.152177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b80f45-2950-486c-98f8-f95f3d6377d3 service swift] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:28.152707 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b80f45-2950-486c-98f8-f95f3d6377d3 service swift] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:28.153920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b80f45-2950-486c-98f8-f95f3d6377d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:28.177253 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b80f45-2950-486c-98f8-f95f3d6377d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:28.194130 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b80f45-2950-486c-98f8-f95f3d6377d3 service swift] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:28.205773 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1139/2282] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:10:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:10:29.315309 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd34692-266e-442d-831b-f85b735558b7 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:29.316351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd34692-266e-442d-831b-f85b735558b7 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:29.316833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd34692-266e-442d-831b-f85b735558b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:29.338846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-7fd34692-266e-442d-831b-f85b735558b7 None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:29.340364 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-7fd34692-266e-442d-831b-f85b735558b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:29.349630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1144/2283] 10.222.0.27 () {58 vars in 967 bytes} [Thu Sep 8 16:10:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:29.370307 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b80f45-2950-486c-98f8-f95f3d6377d3 service swift] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:29.400134 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b80f45-2950-486c-98f8-f95f3d6377d3 service swift] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:29.413661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2bc48-594e-4ef3-ae72-20ce649f7c0e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:29.414310 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2bc48-594e-4ef3-ae72-20ce649f7c0e service swift] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:29.414565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2bc48-594e-4ef3-ae72-20ce649f7c0e service swift] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:29.415067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2bc48-594e-4ef3-ae72-20ce649f7c0e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:29.425409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2bc48-594e-4ef3-ae72-20ce649f7c0e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:29.427275 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2bc48-594e-4ef3-ae72-20ce649f7c0e service swift] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:29.438330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1140/2284] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:10:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:10:29.932293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af999a-210c-4710-9ca8-6822daa55369 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:29.932705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af999a-210c-4710-9ca8-6822daa55369 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:29.933157 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af999a-210c-4710-9ca8-6822daa55369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:29.962494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-f9af999a-210c-4710-9ca8-6822daa55369 None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:29.962494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f9af999a-210c-4710-9ca8-6822daa55369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:29.977291 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1145/2285] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Sep 8 16:10:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:29.987881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2bc48-594e-4ef3-ae72-20ce649f7c0e service swift] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:29.999079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2bc48-594e-4ef3-ae72-20ce649f7c0e service swift] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:30.021067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57821d9d-e436-4ad2-9d3c-96825f525336 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:30.021974 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57821d9d-e436-4ad2-9d3c-96825f525336 service swift] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:30.022400 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57821d9d-e436-4ad2-9d3c-96825f525336 service swift] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:30.022825 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57821d9d-e436-4ad2-9d3c-96825f525336 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:30.032833 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57821d9d-e436-4ad2-9d3c-96825f525336 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:30.035140 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57821d9d-e436-4ad2-9d3c-96825f525336 service swift] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:30.059205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1141/2286] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:10:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:10:30.795694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551168c-651a-4507-8d20-94645ca6890b None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:30.797729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551168c-651a-4507-8d20-94645ca6890b None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:30.798197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6551168c-651a-4507-8d20-94645ca6890b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:30.856847 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-6551168c-651a-4507-8d20-94645ca6890b None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:30.861241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-6551168c-651a-4507-8d20-94645ca6890b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:30.884125 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1146/2287] 10.222.0.27 () {58 vars in 967 bytes} [Thu Sep 8 16:10:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:10:30.916170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57821d9d-e436-4ad2-9d3c-96825f525336 service swift] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:30.930753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57821d9d-e436-4ad2-9d3c-96825f525336 service swift] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:30.951850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975b4ef-ae2a-4fe7-a33b-d31c4bc4cabc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:30.952910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975b4ef-ae2a-4fe7-a33b-d31c4bc4cabc service swift] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:30.953439 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975b4ef-ae2a-4fe7-a33b-d31c4bc4cabc service swift] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:30.953937 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975b4ef-ae2a-4fe7-a33b-d31c4bc4cabc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:30.979076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975b4ef-ae2a-4fe7-a33b-d31c4bc4cabc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:30.982858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975b4ef-ae2a-4fe7-a33b-d31c4bc4cabc service swift] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:31.000741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1142/2288] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:10:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:10:33.162269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26922a-b655-40aa-bfcc-740b8415df52 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:33.163124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26922a-b655-40aa-bfcc-740b8415df52 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:33.163703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26922a-b655-40aa-bfcc-740b8415df52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:33.191593 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:10:33.244647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-bd26922a-b655-40aa-bfcc-740b8415df52 None None] MFA Rules not processed for user `02cddad0448d45bb9458d5cc7cdfbc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:33.253775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-bd26922a-b655-40aa-bfcc-740b8415df52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:33.462380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1147/2289] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:10:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:33.507867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975b4ef-ae2a-4fe7-a33b-d31c4bc4cabc service swift] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:33.509974 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8975b4ef-ae2a-4fe7-a33b-d31c4bc4cabc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:33.517517 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:10:33.656793 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975b4ef-ae2a-4fe7-a33b-d31c4bc4cabc service swift] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:33.663113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664db376-214a-445c-a2b3-b1d9707ed9f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:33.663989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664db376-214a-445c-a2b3-b1d9707ed9f3 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:33.664436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664db376-214a-445c-a2b3-b1d9707ed9f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:33.664855 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664db376-214a-445c-a2b3-b1d9707ed9f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:33.847248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664db376-214a-445c-a2b3-b1d9707ed9f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:33.849244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664db376-214a-445c-a2b3-b1d9707ed9f3 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:33.893480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1143/2290] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:10:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:10:34.308723 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26922a-b655-40aa-bfcc-740b8415df52 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:34.310446 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-bd26922a-b655-40aa-bfcc-740b8415df52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:34.463896 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26922a-b655-40aa-bfcc-740b8415df52 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:34.471261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c487cc2-658a-4768-95c0-b194cfe71fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:34.472887 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c487cc2-658a-4768-95c0-b194cfe71fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:34.473472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c487cc2-658a-4768-95c0-b194cfe71fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:34.474057 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c487cc2-658a-4768-95c0-b194cfe71fda admin admin] PATH_INFO: `/v3/users/f66138a7d2594933b09e1fcc7a4c2f1a` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:34.475536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c487cc2-658a-4768-95c0-b194cfe71fda admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66138a7d2594933b09e1fcc7a4c2f1a)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:34.478084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c487cc2-658a-4768-95c0-b194cfe71fda admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:34.622638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2c487cc2-658a-4768-95c0-b194cfe71fda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66138a7d2594933b09e1fcc7a4c2f1a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:34.638987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2c487cc2-658a-4768-95c0-b194cfe71fda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66138a7d2594933b09e1fcc7a4c2f1a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:34.639947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2c487cc2-658a-4768-95c0-b194cfe71fda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66138a7d2594933b09e1fcc7a4c2f1a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:34.640854 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2c487cc2-658a-4768-95c0-b194cfe71fda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66138a7d2594933b09e1fcc7a4c2f1a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:34.655414 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2c487cc2-658a-4768-95c0-b194cfe71fda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66138a7d2594933b09e1fcc7a4c2f1a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:34.693367 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1148/2291] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:10:34 2022] DELETE /identity/v3/users/f66138a7d2594933b09e1fcc7a4c2f1a => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:10:34.698893 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c57fa8-4a9b-4739-ad85-00075f26c1a0 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:34.699934 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c57fa8-4a9b-4739-ad85-00075f26c1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:34.700438 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c57fa8-4a9b-4739-ad85-00075f26c1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:34.751541 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-39c57fa8-4a9b-4739-ad85-00075f26c1a0 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:34.753307 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-39c57fa8-4a9b-4739-ad85-00075f26c1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:34.927015 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1144/2292] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:34.935505 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c487cc2-658a-4768-95c0-b194cfe71fda admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:34.937295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-2c487cc2-658a-4768-95c0-b194cfe71fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:34.960798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:10:35.044212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c487cc2-658a-4768-95c0-b194cfe71fda admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:35.048174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca26ff18-19de-40aa-99f9-9127f8f1c491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:35.048803 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26ff18-19de-40aa-99f9-9127f8f1c491 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:35.049083 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26ff18-19de-40aa-99f9-9127f8f1c491 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:35.049355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca26ff18-19de-40aa-99f9-9127f8f1c491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:35.060223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26ff18-19de-40aa-99f9-9127f8f1c491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:35.062089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca26ff18-19de-40aa-99f9-9127f8f1c491 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:35.079694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1149/2293] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:10:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:10:36.281708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c57fa8-4a9b-4739-ad85-00075f26c1a0 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:36.283739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-39c57fa8-4a9b-4739-ad85-00075f26c1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:36.310362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c57fa8-4a9b-4739-ad85-00075f26c1a0 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:36.326909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69826848-fef1-460f-9aff-3c846e56bbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:36.328993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69826848-fef1-460f-9aff-3c846e56bbe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:36.331473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69826848-fef1-460f-9aff-3c846e56bbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:36.333933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69826848-fef1-460f-9aff-3c846e56bbe9 admin admin] PATH_INFO: `/v3/projects/19a3e27d765147298f6d421fc614f77f` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:36.338718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69826848-fef1-460f-9aff-3c846e56bbe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a3e27d765147298f6d421fc614f77f)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:36.342948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69826848-fef1-460f-9aff-3c846e56bbe9 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:36.428164 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:10:36.449226 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-69826848-fef1-460f-9aff-3c846e56bbe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a3e27d765147298f6d421fc614f77f', 'request_id': 'req-69826848-fef1-460f-9aff-3c846e56bbe9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:36.471089 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-69826848-fef1-460f-9aff-3c846e56bbe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a3e27d765147298f6d421fc614f77f', 'request_id': 'req-69826848-fef1-460f-9aff-3c846e56bbe9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:36.486832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-69826848-fef1-460f-9aff-3c846e56bbe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a3e27d765147298f6d421fc614f77f', 'request_id': 'req-69826848-fef1-460f-9aff-3c846e56bbe9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:36.524916 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-69826848-fef1-460f-9aff-3c846e56bbe9 admin admin] The token cache is being invalidate because project 19a3e27d765147298f6d421fc614f77f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:10:36.529617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-69826848-fef1-460f-9aff-3c846e56bbe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:36.530866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-69826848-fef1-460f-9aff-3c846e56bbe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:10:36.533363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1145/2294] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:10:36 2022] DELETE /identity/v3/projects/19a3e27d765147298f6d421fc614f77f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:10:45.069975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=22 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:10:46.500706 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:10:53.803129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be368b20-7222-4724-8f3a-58e0aaa6c27d None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:53.803765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be368b20-7222-4724-8f3a-58e0aaa6c27d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:53.803765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be368b20-7222-4724-8f3a-58e0aaa6c27d None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:53.805447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1150/2295] 10.222.0.27 () {54 vars in 921 bytes} [Thu Sep 8 16:10:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:10:53.812874 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0ac7e-a069-4afe-8e2a-11959fe10ab8 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:53.813403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0ac7e-a069-4afe-8e2a-11959fe10ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:53.813403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0ac7e-a069-4afe-8e2a-11959fe10ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:53.841729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-14c0ac7e-a069-4afe-8e2a-11959fe10ab8 None None] MFA Rules not processed for user `6e9209e77ce44b128e25a8ddcf78f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:53.841729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-14c0ac7e-a069-4afe-8e2a-11959fe10ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:53.914995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1146/2296] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Sep 8 16:10:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:10:53.928447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be368b20-7222-4724-8f3a-58e0aaa6c27d None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:53.930360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-be368b20-7222-4724-8f3a-58e0aaa6c27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:53.993740 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be368b20-7222-4724-8f3a-58e0aaa6c27d None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:53.998240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70488f40-a0fb-4dae-982c-1d3d2c7f2ccf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f24ee16c043244938901c4e850dea967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:53.998862 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70488f40-a0fb-4dae-982c-1d3d2c7f2ccf service glance] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:53.999135 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70488f40-a0fb-4dae-982c-1d3d2c7f2ccf service glance] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:53.999388 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70488f40-a0fb-4dae-982c-1d3d2c7f2ccf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:54.010716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70488f40-a0fb-4dae-982c-1d3d2c7f2ccf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:54.012214 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70488f40-a0fb-4dae-982c-1d3d2c7f2ccf service glance] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:54.023061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1151/2297] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:10:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:10:54.105526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3b45a-0f45-434b-b95e-64c2d8f40339 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:54.105730 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3b45a-0f45-434b-b95e-64c2d8f40339 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:54.105975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f3b45a-0f45-434b-b95e-64c2d8f40339 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:54.107332 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1147/2298] 10.222.0.27 () {54 vars in 921 bytes} [Thu Sep 8 16:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:10:54.114351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e4e32-7bff-40f6-a64b-0970b268ba98 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:54.114640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e4e32-7bff-40f6-a64b-0970b268ba98 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:54.114909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738e4e32-7bff-40f6-a64b-0970b268ba98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:54.137017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-738e4e32-7bff-40f6-a64b-0970b268ba98 None None] MFA Rules not processed for user `3aae65f9ec394f97aeafad43932dc49c`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:54.138222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-738e4e32-7bff-40f6-a64b-0970b268ba98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:54.191826 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1152/2299] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Sep 8 16:10:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:54.206553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3b45a-0f45-434b-b95e-64c2d8f40339 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:54.208577 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-89f3b45a-0f45-434b-b95e-64c2d8f40339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:54.222757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f3b45a-0f45-434b-b95e-64c2d8f40339 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:54.226700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5c2f05-4069-4195-8082-9ff047f3f908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:54.227397 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c2f05-4069-4195-8082-9ff047f3f908 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:54.227705 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c2f05-4069-4195-8082-9ff047f3f908 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:54.227974 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5c2f05-4069-4195-8082-9ff047f3f908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:54.235223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c2f05-4069-4195-8082-9ff047f3f908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:54.236763 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5c2f05-4069-4195-8082-9ff047f3f908 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:54.245299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1148/2300] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:10:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:10:54.362812 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec134e-6935-44bb-b515-56e9fc6b6619 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:54.363106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec134e-6935-44bb-b515-56e9fc6b6619 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:54.363544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec134e-6935-44bb-b515-56e9fc6b6619 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:54.364978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1153/2301] 10.222.0.27 () {54 vars in 921 bytes} [Thu Sep 8 16:10:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:10:54.370731 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d49792-cd2d-4e19-8aa2-2521999e8887 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:54.371028 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d49792-cd2d-4e19-8aa2-2521999e8887 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:54.371295 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d49792-cd2d-4e19-8aa2-2521999e8887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:54.392520 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-79d49792-cd2d-4e19-8aa2-2521999e8887 None None] MFA Rules not processed for user `6e9209e77ce44b128e25a8ddcf78f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:54.393721 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-79d49792-cd2d-4e19-8aa2-2521999e8887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:54.404946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1149/2302] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Sep 8 16:10:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:10:54.415709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec134e-6935-44bb-b515-56e9fc6b6619 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:10:54.427852 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec134e-6935-44bb-b515-56e9fc6b6619 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:10:54.431640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8f4e9-93a1-462c-8eb9-c353f87f9732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:10:54.432290 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8f4e9-93a1-462c-8eb9-c353f87f9732 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:54.432567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8f4e9-93a1-462c-8eb9-c353f87f9732 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:54.432835 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8f4e9-93a1-462c-8eb9-c353f87f9732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:54.443941 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8f4e9-93a1-462c-8eb9-c353f87f9732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:10:54.445374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8f4e9-93a1-462c-8eb9-c353f87f9732 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:10:54.456030 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1154/2303] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:10:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:10:59.913688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36e463-e1fb-4b2f-be7e-b1358994243b None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:10:59.914226 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36e463-e1fb-4b2f-be7e-b1358994243b None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:10:59.914226 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e36e463-e1fb-4b2f-be7e-b1358994243b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:10:59.934091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-5e36e463-e1fb-4b2f-be7e-b1358994243b None None] MFA Rules not processed for user `77aeaed3d55945e49b0eb6e3c0519294`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:10:59.935308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5e36e463-e1fb-4b2f-be7e-b1358994243b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:10:59.993401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1150/2304] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:10:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:11:00.846941 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547b446-ad18-480e-a43b-984bbd3c0093 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:00.847379 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547b446-ad18-480e-a43b-984bbd3c0093 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:00.847659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9547b446-ad18-480e-a43b-984bbd3c0093 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:00.849312 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1155/2305] 10.222.0.27 () {54 vars in 961 bytes} [Thu Sep 8 16:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:11:00.856266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627f729-bae2-4657-b579-ab07739a32ac None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:00.856639 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627f729-bae2-4657-b579-ab07739a32ac None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:00.856939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627f729-bae2-4657-b579-ab07739a32ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:00.880533 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f627f729-bae2-4657-b579-ab07739a32ac None None] MFA Rules not processed for user `dd28bb31b55e45668212c9ab69ed3ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:11:00.882033 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f627f729-bae2-4657-b579-ab07739a32ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:00.954130 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1151/2306] 10.222.0.27 () {58 vars in 1058 bytes} [Thu Sep 8 16:11:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:11:00.962735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547b446-ad18-480e-a43b-984bbd3c0093 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:00.981393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9547b446-ad18-480e-a43b-984bbd3c0093 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:00.987773 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b352ae-c681-4bcb-b59e-022d376f3c31 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd28bb31b55e45668212c9ab69ed3ac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:00.988414 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b352ae-c681-4bcb-b59e-022d376f3c31 service placement] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:00.988655 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b352ae-c681-4bcb-b59e-022d376f3c31 service placement] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:00.988937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b352ae-c681-4bcb-b59e-022d376f3c31 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:00.990720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a9b352ae-c681-4bcb-b59e-022d376f3c31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:01.003817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b352ae-c681-4bcb-b59e-022d376f3c31 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:01.005440 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b352ae-c681-4bcb-b59e-022d376f3c31 service placement] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:01.023850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1156/2307] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:11:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:11:08.342700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06fca7-b949-44ad-8b88-30e25449eefd None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:08.343238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06fca7-b949-44ad-8b88-30e25449eefd None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:08.343238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06fca7-b949-44ad-8b88-30e25449eefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:08.362755 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-6f06fca7-b949-44ad-8b88-30e25449eefd None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:11:08.363993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6f06fca7-b949-44ad-8b88-30e25449eefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:08.372565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:11:08.421899 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1152/2308] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Sep 8 16:11:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:11:08.434184 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b352ae-c681-4bcb-b59e-022d376f3c31 service placement] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:08.435635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a9b352ae-c681-4bcb-b59e-022d376f3c31 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:08.508696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b352ae-c681-4bcb-b59e-022d376f3c31 service placement] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:08.527563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e1031-95c9-46e3-ab7b-7f19e283e71b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:08.527563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e1031-95c9-46e3-ab7b-7f19e283e71b service swift] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:08.527563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e1031-95c9-46e3-ab7b-7f19e283e71b service swift] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:08.527563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e1031-95c9-46e3-ab7b-7f19e283e71b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:08.536651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e1031-95c9-46e3-ab7b-7f19e283e71b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:08.537673 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e1031-95c9-46e3-ab7b-7f19e283e71b service swift] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:08.550731 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1157/2309] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:11:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:11:18.414594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=19 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:11:18.543067 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=93 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:11:29.733864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06fca7-b949-44ad-8b88-30e25449eefd None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:29.735496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6f06fca7-b949-44ad-8b88-30e25449eefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:29.747118 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06fca7-b949-44ad-8b88-30e25449eefd None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:29.751039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097d18b-33bd-464f-bf68-1d83da1f808f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:29.751708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097d18b-33bd-464f-bf68-1d83da1f808f service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:29.751986 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097d18b-33bd-464f-bf68-1d83da1f808f service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:29.752298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0097d18b-33bd-464f-bf68-1d83da1f808f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:29.754123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0097d18b-33bd-464f-bf68-1d83da1f808f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:29.815222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097d18b-33bd-464f-bf68-1d83da1f808f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:29.817091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0097d18b-33bd-464f-bf68-1d83da1f808f service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:29.826579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1153/2310] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:11:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:11:32.866957 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e1031-95c9-46e3-ab7b-7f19e283e71b service swift] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:32.868713 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-dc3e1031-95c9-46e3-ab7b-7f19e283e71b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:32.939213 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e1031-95c9-46e3-ab7b-7f19e283e71b service swift] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:32.944537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e49659-88ef-474d-83bc-d01d22f7e4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:32.945122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e49659-88ef-474d-83bc-d01d22f7e4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:32.945397 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e49659-88ef-474d-83bc-d01d22f7e4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:32.945712 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e49659-88ef-474d-83bc-d01d22f7e4cf admin admin] PATH_INFO: `/v3/users/2127487c50884141b2776f05e96471fd` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:32.946832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e49659-88ef-474d-83bc-d01d22f7e4cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2127487c50884141b2776f05e96471fd)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:32.948811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e49659-88ef-474d-83bc-d01d22f7e4cf admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:32.998503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-31e49659-88ef-474d-83bc-d01d22f7e4cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2127487c50884141b2776f05e96471fd'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:11:33.006916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-31e49659-88ef-474d-83bc-d01d22f7e4cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2127487c50884141b2776f05e96471fd'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:11:33.007634 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-31e49659-88ef-474d-83bc-d01d22f7e4cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2127487c50884141b2776f05e96471fd'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:11:33.008341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-31e49659-88ef-474d-83bc-d01d22f7e4cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2127487c50884141b2776f05e96471fd'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:11:33.015617 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-31e49659-88ef-474d-83bc-d01d22f7e4cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2127487c50884141b2776f05e96471fd'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:11:33.032769 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1158/2311] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:11:32 2022] DELETE /identity/v3/users/2127487c50884141b2776f05e96471fd => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:11:33.211934 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097d18b-33bd-464f-bf68-1d83da1f808f service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:33.224537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0097d18b-33bd-464f-bf68-1d83da1f808f service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:33.283709 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04258ea1-9cdb-4d66-8cb6-a61f55b444e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:33.284360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04258ea1-9cdb-4d66-8cb6-a61f55b444e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:33.284635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04258ea1-9cdb-4d66-8cb6-a61f55b444e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:33.284906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04258ea1-9cdb-4d66-8cb6-a61f55b444e5 admin admin] PATH_INFO: `/v3/projects/bafdc72abf95401f8784411bc8e55822` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:33.286079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04258ea1-9cdb-4d66-8cb6-a61f55b444e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bafdc72abf95401f8784411bc8e55822)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:33.288033 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04258ea1-9cdb-4d66-8cb6-a61f55b444e5 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:33.339147 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-04258ea1-9cdb-4d66-8cb6-a61f55b444e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bafdc72abf95401f8784411bc8e55822', 'request_id': 'req-04258ea1-9cdb-4d66-8cb6-a61f55b444e5'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:11:33.349604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-04258ea1-9cdb-4d66-8cb6-a61f55b444e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bafdc72abf95401f8784411bc8e55822', 'request_id': 'req-04258ea1-9cdb-4d66-8cb6-a61f55b444e5'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:11:33.356098 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-04258ea1-9cdb-4d66-8cb6-a61f55b444e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bafdc72abf95401f8784411bc8e55822', 'request_id': 'req-04258ea1-9cdb-4d66-8cb6-a61f55b444e5'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:11:33.367805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-04258ea1-9cdb-4d66-8cb6-a61f55b444e5 admin admin] The token cache is being invalidate because project bafdc72abf95401f8784411bc8e55822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:11:33.368091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-04258ea1-9cdb-4d66-8cb6-a61f55b444e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:11:33.368728 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-04258ea1-9cdb-4d66-8cb6-a61f55b444e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:11:33.370226 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1154/2312] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:11:33 2022] DELETE /identity/v3/projects/bafdc72abf95401f8784411bc8e55822 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:11:33.375489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e10d7-11e7-41c2-a8d7-84029c4d258e None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:33.375809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e10d7-11e7-41c2-a8d7-84029c4d258e None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:33.376080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e10d7-11e7-41c2-a8d7-84029c4d258e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:33.396750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-956e10d7-11e7-41c2-a8d7-84029c4d258e None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:11:33.397977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-956e10d7-11e7-41c2-a8d7-84029c4d258e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:33.465104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1159/2313] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:11:33.474265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d68c47-a236-4185-912c-2db840750a5e None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:33.474549 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d68c47-a236-4185-912c-2db840750a5e None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:33.474845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d68c47-a236-4185-912c-2db840750a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:33.505660 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-73d68c47-a236-4185-912c-2db840750a5e None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:11:33.506879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-73d68c47-a236-4185-912c-2db840750a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:33.516565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1155/2314] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:11:33.521819 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e10d7-11e7-41c2-a8d7-84029c4d258e None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:33.532726 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e10d7-11e7-41c2-a8d7-84029c4d258e None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:33.541497 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642063b7-9979-4ea1-810f-e58e0ca866e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:33.542319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642063b7-9979-4ea1-810f-e58e0ca866e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:33.542595 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642063b7-9979-4ea1-810f-e58e0ca866e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:33.542856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642063b7-9979-4ea1-810f-e58e0ca866e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:33.543685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642063b7-9979-4ea1-810f-e58e0ca866e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:11:33.544020 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642063b7-9979-4ea1-810f-e58e0ca866e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:33.546372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642063b7-9979-4ea1-810f-e58e0ca866e4 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:33.563340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1160/2315] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:11:33.568398 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d68c47-a236-4185-912c-2db840750a5e None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:33.577890 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d68c47-a236-4185-912c-2db840750a5e None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:33.583880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c48698-d9df-430b-badc-75d2e402dc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:33.584501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c48698-d9df-430b-badc-75d2e402dc89 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:33.584800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c48698-d9df-430b-badc-75d2e402dc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:33.585072 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c48698-d9df-430b-badc-75d2e402dc89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:33.585744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c48698-d9df-430b-badc-75d2e402dc89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:33.587767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c48698-d9df-430b-badc-75d2e402dc89 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:33.605204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:11:33.621107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1156/2316] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:11:33 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:11:33.625485 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642063b7-9979-4ea1-810f-e58e0ca866e4 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:33.636263 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642063b7-9979-4ea1-810f-e58e0ca866e4 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:33.640589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:11:33.699780 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5d5a8-148f-4bdf-8b36-4e58b302be36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:33.700390 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5d5a8-148f-4bdf-8b36-4e58b302be36 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:33.700660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5d5a8-148f-4bdf-8b36-4e58b302be36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:33.701036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5d5a8-148f-4bdf-8b36-4e58b302be36 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:33.701682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5d5a8-148f-4bdf-8b36-4e58b302be36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:33.703616 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5d5a8-148f-4bdf-8b36-4e58b302be36 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:33.731530 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1161/2317] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:11:33 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:11:33.732634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c48698-d9df-430b-badc-75d2e402dc89 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:33.743404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c48698-d9df-430b-badc-75d2e402dc89 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:33.748091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dece4e9-6315-474e-a224-f918c235d811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:33.748696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dece4e9-6315-474e-a224-f918c235d811 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:33.748972 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dece4e9-6315-474e-a224-f918c235d811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:33.749244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dece4e9-6315-474e-a224-f918c235d811 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:33.749882 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dece4e9-6315-474e-a224-f918c235d811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:33.751520 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dece4e9-6315-474e-a224-f918c235d811 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:33.761736 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1157/2318] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:11:33.765797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5d5a8-148f-4bdf-8b36-4e58b302be36 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:33.776134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5d5a8-148f-4bdf-8b36-4e58b302be36 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:33.781291 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f352f-45b7-4c2d-a9be-f0fdac270316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:33.781907 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f352f-45b7-4c2d-a9be-f0fdac270316 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:33.782234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f352f-45b7-4c2d-a9be-f0fdac270316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:33.782509 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f352f-45b7-4c2d-a9be-f0fdac270316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:33.783236 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f352f-45b7-4c2d-a9be-f0fdac270316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:33.784978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f352f-45b7-4c2d-a9be-f0fdac270316 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:33.800857 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1162/2319] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:11:33.802369 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dece4e9-6315-474e-a224-f918c235d811 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:33.819252 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dece4e9-6315-474e-a224-f918c235d811 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:33.827248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fb7d6-a5ba-46b0-9cfc-c8596e6cc4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:33.828593 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fb7d6-a5ba-46b0-9cfc-c8596e6cc4c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:33.828873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fb7d6-a5ba-46b0-9cfc-c8596e6cc4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:33.829143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2fb7d6-a5ba-46b0-9cfc-c8596e6cc4c4 admin admin] PATH_INFO: `/v3/projects/2245b7557087417bb8a32322c8673cef/users/f3be588325bf40beae354c1237d8e9ea/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:33.859566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fb7d6-a5ba-46b0-9cfc-c8596e6cc4c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2245b7557087417bb8a32322c8673cef, user_id=f3be588325bf40beae354c1237d8e9ea, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:33.864729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2fb7d6-a5ba-46b0-9cfc-c8596e6cc4c4 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:33.886665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1158/2320] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:11:33 2022] PUT /identity/v3/projects/2245b7557087417bb8a32322c8673cef/users/f3be588325bf40beae354c1237d8e9ea/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:11:33.894241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8dc5fc-b347-482d-94a3-808bcf1cf6a0 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:33.894756 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8dc5fc-b347-482d-94a3-808bcf1cf6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:33.894756 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8dc5fc-b347-482d-94a3-808bcf1cf6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:33.923346 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-ad8dc5fc-b347-482d-94a3-808bcf1cf6a0 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:11:33.924685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ad8dc5fc-b347-482d-94a3-808bcf1cf6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:33.985451 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1163/2321] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:11:33.995650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fb7d6-a5ba-46b0-9cfc-c8596e6cc4c4 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:33.997712 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6f2fb7d6-a5ba-46b0-9cfc-c8596e6cc4c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:34.055887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2fb7d6-a5ba-46b0-9cfc-c8596e6cc4c4 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:34.059842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e253e-75aa-4328-8257-88a3aa8dbde0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:34.060442 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e253e-75aa-4328-8257-88a3aa8dbde0 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:34.060722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e253e-75aa-4328-8257-88a3aa8dbde0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:34.060994 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e253e-75aa-4328-8257-88a3aa8dbde0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:34.071162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e253e-75aa-4328-8257-88a3aa8dbde0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:34.072661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e253e-75aa-4328-8257-88a3aa8dbde0 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:34.082864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1159/2322] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:11:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:11:37.461475 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef02748-1953-47ab-b7d4-f013c0123133 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:37.461914 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef02748-1953-47ab-b7d4-f013c0123133 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:37.462023 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef02748-1953-47ab-b7d4-f013c0123133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:37.489197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-9ef02748-1953-47ab-b7d4-f013c0123133 None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:11:37.490788 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9ef02748-1953-47ab-b7d4-f013c0123133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:37.563016 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1164/2323] 10.222.0.27 () {58 vars in 967 bytes} [Thu Sep 8 16:11:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:11:37.595675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e253e-75aa-4328-8257-88a3aa8dbde0 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:37.597277 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0f9e253e-75aa-4328-8257-88a3aa8dbde0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:37.703693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e253e-75aa-4328-8257-88a3aa8dbde0 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:37.719896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa17e85-26fa-45c0-aaac-2ccbf102b2da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:37.721103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa17e85-26fa-45c0-aaac-2ccbf102b2da service swift] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:37.721567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa17e85-26fa-45c0-aaac-2ccbf102b2da service swift] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:37.722321 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa17e85-26fa-45c0-aaac-2ccbf102b2da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:37.731751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa17e85-26fa-45c0-aaac-2ccbf102b2da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:37.734146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa17e85-26fa-45c0-aaac-2ccbf102b2da service swift] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:37.748093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1160/2324] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:11:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:11:38.420549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497919c7-b63d-4bbd-bcfb-53eae3adbaa7 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:38.420957 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497919c7-b63d-4bbd-bcfb-53eae3adbaa7 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:38.421295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497919c7-b63d-4bbd-bcfb-53eae3adbaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:38.465580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-497919c7-b63d-4bbd-bcfb-53eae3adbaa7 None None] MFA Rules not processed for user `f3be588325bf40beae354c1237d8e9ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:11:38.466897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-497919c7-b63d-4bbd-bcfb-53eae3adbaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:38.495200 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:11:38.552473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1165/2325] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:11:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:11:38.556858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa17e85-26fa-45c0-aaac-2ccbf102b2da service swift] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:38.568379 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa17e85-26fa-45c0-aaac-2ccbf102b2da service swift] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:38.573358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857bf9d-e654-45c3-b436-cf3652a8e534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:38.574155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857bf9d-e654-45c3-b436-cf3652a8e534 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:38.574488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857bf9d-e654-45c3-b436-cf3652a8e534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:38.574820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857bf9d-e654-45c3-b436-cf3652a8e534 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:38.576792 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857bf9d-e654-45c3-b436-cf3652a8e534 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:38.584555 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857bf9d-e654-45c3-b436-cf3652a8e534 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:38.642158 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1161/2326] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:11:38 2022] POST /identity/v3/projects => generated 373 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:11:38.643518 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497919c7-b63d-4bbd-bcfb-53eae3adbaa7 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:38.661327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497919c7-b63d-4bbd-bcfb-53eae3adbaa7 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:38.730048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d59a87-9987-43e0-9297-23af10cb3e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:38.732635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59a87-9987-43e0-9297-23af10cb3e8b admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:38.733597 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59a87-9987-43e0-9297-23af10cb3e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:38.733873 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d59a87-9987-43e0-9297-23af10cb3e8b admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:38.734704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d59a87-9987-43e0-9297-23af10cb3e8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:38.737464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d59a87-9987-43e0-9297-23af10cb3e8b admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:38.778947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1166/2327] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:11:38 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:11:38.783331 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857bf9d-e654-45c3-b436-cf3652a8e534 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:38.797815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857bf9d-e654-45c3-b436-cf3652a8e534 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:38.806444 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895ec19-782b-4772-8c7e-67fa522fe4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:38.807176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895ec19-782b-4772-8c7e-67fa522fe4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:38.807733 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895ec19-782b-4772-8c7e-67fa522fe4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:38.808299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895ec19-782b-4772-8c7e-67fa522fe4c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:38.809164 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895ec19-782b-4772-8c7e-67fa522fe4c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:38.811234 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895ec19-782b-4772-8c7e-67fa522fe4c7 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:38.838763 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1162/2328] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:11:38.843127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d59a87-9987-43e0-9297-23af10cb3e8b admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:38.854376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d59a87-9987-43e0-9297-23af10cb3e8b admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:38.859160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924a38c-9b61-4a94-a274-af283d419676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:38.859901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924a38c-9b61-4a94-a274-af283d419676 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:38.860180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924a38c-9b61-4a94-a274-af283d419676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:38.860455 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924a38c-9b61-4a94-a274-af283d419676 admin admin] PATH_INFO: `/v3/projects/ef47c6c1b6b9478e80b5ec45add04f45/users/f27ce8a59f7a463e9106dd8f6ad0191b/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:38.885164 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924a38c-9b61-4a94-a274-af283d419676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef47c6c1b6b9478e80b5ec45add04f45, user_id=f27ce8a59f7a463e9106dd8f6ad0191b, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:38.888965 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924a38c-9b61-4a94-a274-af283d419676 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:38.911911 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1167/2329] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:11:38 2022] PUT /identity/v3/projects/ef47c6c1b6b9478e80b5ec45add04f45/users/f27ce8a59f7a463e9106dd8f6ad0191b/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:11:38.913357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895ec19-782b-4772-8c7e-67fa522fe4c7 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:38.935079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895ec19-782b-4772-8c7e-67fa522fe4c7 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:38.959500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:11:39.011689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d88488-9e1b-490f-8d12-13f2e2464920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:39.012462 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d88488-9e1b-490f-8d12-13f2e2464920 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:39.012733 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d88488-9e1b-490f-8d12-13f2e2464920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:39.012997 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d88488-9e1b-490f-8d12-13f2e2464920 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:39.013646 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d88488-9e1b-490f-8d12-13f2e2464920 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:39.015278 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d88488-9e1b-490f-8d12-13f2e2464920 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:39.026004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1163/2330] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:11:39.030376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924a38c-9b61-4a94-a274-af283d419676 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:39.041376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924a38c-9b61-4a94-a274-af283d419676 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:39.046162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f154aa-fff2-4931-9aa7-f9e2f95c7f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:39.046922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f154aa-fff2-4931-9aa7-f9e2f95c7f25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:39.047270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f154aa-fff2-4931-9aa7-f9e2f95c7f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:39.047722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f154aa-fff2-4931-9aa7-f9e2f95c7f25 admin admin] PATH_INFO: `/v3/projects/ef47c6c1b6b9478e80b5ec45add04f45/users/f27ce8a59f7a463e9106dd8f6ad0191b/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:39.049787 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f154aa-fff2-4931-9aa7-f9e2f95c7f25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef47c6c1b6b9478e80b5ec45add04f45, user_id=f27ce8a59f7a463e9106dd8f6ad0191b, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:39.052586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f154aa-fff2-4931-9aa7-f9e2f95c7f25 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:39.068539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1168/2331] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:11:39 2022] PUT /identity/v3/projects/ef47c6c1b6b9478e80b5ec45add04f45/users/f27ce8a59f7a463e9106dd8f6ad0191b/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:11:45.718605 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64708d9e-be53-44e6-9224-4b997767ae2e None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:45.719576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64708d9e-be53-44e6-9224-4b997767ae2e None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:45.720249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64708d9e-be53-44e6-9224-4b997767ae2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:45.782031 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-64708d9e-be53-44e6-9224-4b997767ae2e None None] MFA Rules not processed for user `f27ce8a59f7a463e9106dd8f6ad0191b`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:11:45.783860 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-64708d9e-be53-44e6-9224-4b997767ae2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:45.876773 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1164/2332] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:11:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:11:45.890981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f154aa-fff2-4931-9aa7-f9e2f95c7f25 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:11:45.892830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-46f154aa-fff2-4931-9aa7-f9e2f95c7f25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:45.971353 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f154aa-fff2-4931-9aa7-f9e2f95c7f25 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:11:45.976892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93488072-94a6-4735-bdd7-e9422e703298 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:11:45.977515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93488072-94a6-4735-bdd7-e9422e703298 service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:45.977795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93488072-94a6-4735-bdd7-e9422e703298 service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:45.978084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93488072-94a6-4735-bdd7-e9422e703298 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:45.986540 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:11:46.035216 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93488072-94a6-4735-bdd7-e9422e703298 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:11:46.036846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93488072-94a6-4735-bdd7-e9422e703298 service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:11:46.045833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1169/2333] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:11:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:11:47.109838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a3d25-ab3e-4843-8a71-99b531b9d2f4 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:47.110371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a3d25-ab3e-4843-8a71-99b531b9d2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:47.110371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a3d25-ab3e-4843-8a71-99b531b9d2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:47.130498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f12a3d25-ab3e-4843-8a71-99b531b9d2f4 None None] MFA Rules not processed for user `f3be588325bf40beae354c1237d8e9ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:11:47.131744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f12a3d25-ab3e-4843-8a71-99b531b9d2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:47.138918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1165/2334] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:11:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:11:56.041641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=23 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:11:57.124736 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=57 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:11:57.349463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3b3e1-6a70-4d3a-8a98-f92b15c9f572 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:11:57.350003 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3b3e1-6a70-4d3a-8a98-f92b15c9f572 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:11:57.350003 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b3b3e1-6a70-4d3a-8a98-f92b15c9f572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:11:57.371808 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-b9b3b3e1-6a70-4d3a-8a98-f92b15c9f572 None None] MFA Rules not processed for user `00b673d5c65d40d9a90cc3b579e83e7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:11:57.373032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b9b3b3e1-6a70-4d3a-8a98-f92b15c9f572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:11:57.433846 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1170/2335] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:11:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:12:07.426540 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=25 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:12:12.963902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3fe3e-65fc-419b-9c6c-307a89d6c267 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:12:12.964522 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3fe3e-65fc-419b-9c6c-307a89d6c267 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:12:12.964522 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3fe3e-65fc-419b-9c6c-307a89d6c267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:12:12.985262 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-d2e3fe3e-65fc-419b-9c6c-307a89d6c267 None None] MFA Rules not processed for user `18375266a64348df879a9eec53bfcbf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:12:12.986496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d2e3fe3e-65fc-419b-9c6c-307a89d6c267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:12:13.049126 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1166/2336] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:12:12 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:12:23.042579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=25 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:12:25.674326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3b3e1-6a70-4d3a-8a98-f92b15c9f572 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:12:25.675902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b9b3b3e1-6a70-4d3a-8a98-f92b15c9f572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:12:25.735730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b3b3e1-6a70-4d3a-8a98-f92b15c9f572 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:12:25.739295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521aab2d-c5e1-43c8-bb78-70bedc21e8d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:12:25.739930 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521aab2d-c5e1-43c8-bb78-70bedc21e8d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:12:25.740205 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521aab2d-c5e1-43c8-bb78-70bedc21e8d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:12:25.740471 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521aab2d-c5e1-43c8-bb78-70bedc21e8d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:12:25.749652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521aab2d-c5e1-43c8-bb78-70bedc21e8d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:12:25.751104 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521aab2d-c5e1-43c8-bb78-70bedc21e8d6 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:12:25.760582 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1171/2337] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:12:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:12:35.169344 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f8782-6990-4507-bc19-e113e351fb32 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:12:35.169874 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f8782-6990-4507-bc19-e113e351fb32 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:12:35.169874 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f8782-6990-4507-bc19-e113e351fb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:12:35.190208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-4f5f8782-6990-4507-bc19-e113e351fb32 None None] MFA Rules not processed for user `c89859a30a2442b8ae13fe0444782d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:12:35.191414 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-4f5f8782-6990-4507-bc19-e113e351fb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:12:35.248368 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1167/2338] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:12:35 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:12:35.756310 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=26 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:12:45.241843 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=25 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:12:47.715222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521aab2d-c5e1-43c8-bb78-70bedc21e8d6 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:12:47.724835 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521aab2d-c5e1-43c8-bb78-70bedc21e8d6 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:12:47.728818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bddf7-8da8-47f1-98dc-bf7efe694433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:12:47.729407 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bddf7-8da8-47f1-98dc-bf7efe694433 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:12:47.729682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bddf7-8da8-47f1-98dc-bf7efe694433 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:12:47.729944 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bddf7-8da8-47f1-98dc-bf7efe694433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:12:47.731856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0d6bddf7-8da8-47f1-98dc-bf7efe694433 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:12:47.745118 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bddf7-8da8-47f1-98dc-bf7efe694433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:12:47.746526 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bddf7-8da8-47f1-98dc-bf7efe694433 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:12:47.757800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1172/2339] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:12:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:12:47.948384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f8782-6990-4507-bc19-e113e351fb32 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:12:47.958881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f8782-6990-4507-bc19-e113e351fb32 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:12:47.963461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f4882-a1e1-429c-82a7-8cb32dfb3ea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:12:47.964187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f4882-a1e1-429c-82a7-8cb32dfb3ea0 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:12:47.964343 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f4882-a1e1-429c-82a7-8cb32dfb3ea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:12:47.964961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f4882-a1e1-429c-82a7-8cb32dfb3ea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:12:47.967204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-cf3f4882-a1e1-429c-82a7-8cb32dfb3ea0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:12:47.983580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f4882-a1e1-429c-82a7-8cb32dfb3ea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:12:47.985044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3f4882-a1e1-429c-82a7-8cb32dfb3ea0 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:12:47.996496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1168/2340] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:12:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:12:57.753144 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=6 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:12:57.991285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=6 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:06.363131 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bddf7-8da8-47f1-98dc-bf7efe694433 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:06.375381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bddf7-8da8-47f1-98dc-bf7efe694433 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:06.382560 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ead0d-0329-45a3-b5e0-264a3f7bcdb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:06.383198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ead0d-0329-45a3-b5e0-264a3f7bcdb2 service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:06.383475 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ead0d-0329-45a3-b5e0-264a3f7bcdb2 service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:06.383763 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ead0d-0329-45a3-b5e0-264a3f7bcdb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:06.385823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-2f5ead0d-0329-45a3-b5e0-264a3f7bcdb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:06.397329 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ead0d-0329-45a3-b5e0-264a3f7bcdb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:06.398951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ead0d-0329-45a3-b5e0-264a3f7bcdb2 service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:06.408791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1173/2341] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:07.875517 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f4882-a1e1-429c-82a7-8cb32dfb3ea0 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:07.878123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-cf3f4882-a1e1-429c-82a7-8cb32dfb3ea0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:07.899335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f4882-a1e1-429c-82a7-8cb32dfb3ea0 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:07.906621 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f80dc7-add7-4fcc-8a9d-4667212082ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:07.907221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f80dc7-add7-4fcc-8a9d-4667212082ae service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:07.907482 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f80dc7-add7-4fcc-8a9d-4667212082ae service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:07.907736 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f80dc7-add7-4fcc-8a9d-4667212082ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:07.909801 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-24f80dc7-add7-4fcc-8a9d-4667212082ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:07.920288 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f80dc7-add7-4fcc-8a9d-4667212082ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:07.921862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f80dc7-add7-4fcc-8a9d-4667212082ae service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:07.930936 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1169/2342] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:16.403970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=6 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:16.866943 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99489d41-14e7-499b-9c7f-456851d0fada None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:16.867252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99489d41-14e7-499b-9c7f-456851d0fada None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:16.867616 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99489d41-14e7-499b-9c7f-456851d0fada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:16.895432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-99489d41-14e7-499b-9c7f-456851d0fada None None] MFA Rules not processed for user `00b673d5c65d40d9a90cc3b579e83e7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:16.896785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-99489d41-14e7-499b-9c7f-456851d0fada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:16.913088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1174/2343] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:17.234663 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f80dc7-add7-4fcc-8a9d-4667212082ae service nova] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:17.244882 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f80dc7-add7-4fcc-8a9d-4667212082ae service nova] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:17.249087 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e64a3-b61e-4a1e-9aa9-51ee40c1639b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:17.249672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e64a3-b61e-4a1e-9aa9-51ee40c1639b service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:17.249950 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e64a3-b61e-4a1e-9aa9-51ee40c1639b service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:17.250207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01e64a3-b61e-4a1e-9aa9-51ee40c1639b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:17.252087 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b01e64a3-b61e-4a1e-9aa9-51ee40c1639b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:17.262070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e64a3-b61e-4a1e-9aa9-51ee40c1639b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:17.263472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01e64a3-b61e-4a1e-9aa9-51ee40c1639b service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:17.271569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1170/2344] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:17.613471 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99489d41-14e7-499b-9c7f-456851d0fada None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:17.615085 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-99489d41-14e7-499b-9c7f-456851d0fada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:17.689041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99489d41-14e7-499b-9c7f-456851d0fada None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:17.694160 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74465a-5e4a-4f8b-8089-b49d0f914955 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd28bb31b55e45668212c9ab69ed3ac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:17.694772 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74465a-5e4a-4f8b-8089-b49d0f914955 service placement] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:17.695046 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74465a-5e4a-4f8b-8089-b49d0f914955 service placement] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:17.695319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74465a-5e4a-4f8b-8089-b49d0f914955 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:17.697125 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3c74465a-5e4a-4f8b-8089-b49d0f914955 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:17.708602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74465a-5e4a-4f8b-8089-b49d0f914955 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:17.710185 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74465a-5e4a-4f8b-8089-b49d0f914955 service placement] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:17.720082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1175/2345] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:20.616432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e64a3-b61e-4a1e-9aa9-51ee40c1639b service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:20.625563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01e64a3-b61e-4a1e-9aa9-51ee40c1639b service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:20.636469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84321552-7d36-4e17-901c-98774af5eca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:20.638189 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84321552-7d36-4e17-901c-98774af5eca0 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:20.638656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84321552-7d36-4e17-901c-98774af5eca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:20.639550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84321552-7d36-4e17-901c-98774af5eca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:20.646063 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-84321552-7d36-4e17-901c-98774af5eca0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:20.670113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84321552-7d36-4e17-901c-98774af5eca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:20.672610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84321552-7d36-4e17-901c-98774af5eca0 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:20.696637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1171/2346] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:27.715851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=35 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:29.679869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74465a-5e4a-4f8b-8089-b49d0f914955 service placement] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:29.693552 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74465a-5e4a-4f8b-8089-b49d0f914955 service placement] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:29.700370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff353c-bf65-49ae-809e-934f58adca4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:29.700975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff353c-bf65-49ae-809e-934f58adca4f service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:29.701252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff353c-bf65-49ae-809e-934f58adca4f service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:29.701517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff353c-bf65-49ae-809e-934f58adca4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:29.703344 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9cff353c-bf65-49ae-809e-934f58adca4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:29.715831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff353c-bf65-49ae-809e-934f58adca4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:29.717426 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff353c-bf65-49ae-809e-934f58adca4f service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:29.728149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1176/2347] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:30.152602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84321552-7d36-4e17-901c-98774af5eca0 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:30.156680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=19 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:30.162832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84321552-7d36-4e17-901c-98774af5eca0 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:30.167279 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7507529-0c0a-4d06-a91c-ac511dfc7322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:30.168118 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7507529-0c0a-4d06-a91c-ac511dfc7322 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:30.168544 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7507529-0c0a-4d06-a91c-ac511dfc7322 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:30.168975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7507529-0c0a-4d06-a91c-ac511dfc7322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:30.170987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d7507529-0c0a-4d06-a91c-ac511dfc7322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:30.241891 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7507529-0c0a-4d06-a91c-ac511dfc7322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:30.243491 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7507529-0c0a-4d06-a91c-ac511dfc7322 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:30.254079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1172/2348] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:31.768598 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a5615-9b27-40d3-877e-08c9d03b24db None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:31.769070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a5615-9b27-40d3-877e-08c9d03b24db None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:31.769179 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a5615-9b27-40d3-877e-08c9d03b24db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:31.790037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-301a5615-9b27-40d3-877e-08c9d03b24db None None] MFA Rules not processed for user `965cbbdb06c248698ef54527c29767df`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:31.791163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-301a5615-9b27-40d3-877e-08c9d03b24db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:31.851364 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1177/2349] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:33.456312 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7507529-0c0a-4d06-a91c-ac511dfc7322 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:33.484165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7507529-0c0a-4d06-a91c-ac511dfc7322 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:33.500143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e870e27-5b24-4e80-8737-23a5527df0f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:33.501307 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e870e27-5b24-4e80-8737-23a5527df0f5 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:33.501811 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e870e27-5b24-4e80-8737-23a5527df0f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:33.502309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e870e27-5b24-4e80-8737-23a5527df0f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:33.504795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8e870e27-5b24-4e80-8737-23a5527df0f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:33.523943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e870e27-5b24-4e80-8737-23a5527df0f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:33.526905 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e870e27-5b24-4e80-8737-23a5527df0f5 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:33.541529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1173/2350] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:34.152268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a5615-9b27-40d3-877e-08c9d03b24db None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:34.189177 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a5615-9b27-40d3-877e-08c9d03b24db None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:34.197629 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11b6a1-fa3c-4c05-ab1d-e3fe5a83baa2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd28bb31b55e45668212c9ab69ed3ac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:34.198539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11b6a1-fa3c-4c05-ab1d-e3fe5a83baa2 service placement] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:34.198942 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11b6a1-fa3c-4c05-ab1d-e3fe5a83baa2 service placement] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:34.199349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11b6a1-fa3c-4c05-ab1d-e3fe5a83baa2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:34.201804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9f11b6a1-fa3c-4c05-ab1d-e3fe5a83baa2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:34.220094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11b6a1-fa3c-4c05-ab1d-e3fe5a83baa2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:34.220094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11b6a1-fa3c-4c05-ab1d-e3fe5a83baa2 service placement] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:34.221129 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e870e27-5b24-4e80-8737-23a5527df0f5 service nova] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:34.226527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8e870e27-5b24-4e80-8737-23a5527df0f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:34.230657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1178/2351] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:34.249316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e870e27-5b24-4e80-8737-23a5527df0f5 service nova] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:34.254049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51613e40-2798-455f-ab91-a163a1e0bf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:34.254694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51613e40-2798-455f-ab91-a163a1e0bf65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:34.255132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51613e40-2798-455f-ab91-a163a1e0bf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:34.255408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51613e40-2798-455f-ab91-a163a1e0bf65 admin admin] PATH_INFO: `/v3/users/18375266a64348df879a9eec53bfcbf9` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:34.256600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51613e40-2798-455f-ab91-a163a1e0bf65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18375266a64348df879a9eec53bfcbf9)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:34.258542 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51613e40-2798-455f-ab91-a163a1e0bf65 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:34.305526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-51613e40-2798-455f-ab91-a163a1e0bf65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18375266a64348df879a9eec53bfcbf9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:34.313628 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-51613e40-2798-455f-ab91-a163a1e0bf65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18375266a64348df879a9eec53bfcbf9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:34.314260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-51613e40-2798-455f-ab91-a163a1e0bf65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18375266a64348df879a9eec53bfcbf9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:34.314853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-51613e40-2798-455f-ab91-a163a1e0bf65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18375266a64348df879a9eec53bfcbf9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:34.332893 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-51613e40-2798-455f-ab91-a163a1e0bf65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18375266a64348df879a9eec53bfcbf9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:34.345891 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1174/2352] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:13:34 2022] DELETE /identity/v3/users/18375266a64348df879a9eec53bfcbf9 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:34.575279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11b6a1-fa3c-4c05-ab1d-e3fe5a83baa2 service placement] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:34.588988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11b6a1-fa3c-4c05-ab1d-e3fe5a83baa2 service placement] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:34.649345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dec4d8-c911-4421-9c8e-26a538f75059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:34.649993 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dec4d8-c911-4421-9c8e-26a538f75059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:34.650271 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dec4d8-c911-4421-9c8e-26a538f75059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:34.650538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dec4d8-c911-4421-9c8e-26a538f75059 admin admin] PATH_INFO: `/v3/projects/73c9cf0f579144eb80c090456d363a05` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:34.651686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dec4d8-c911-4421-9c8e-26a538f75059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73c9cf0f579144eb80c090456d363a05)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:34.653586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dec4d8-c911-4421-9c8e-26a538f75059 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:34.702526 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-d8dec4d8-c911-4421-9c8e-26a538f75059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73c9cf0f579144eb80c090456d363a05', 'request_id': 'req-d8dec4d8-c911-4421-9c8e-26a538f75059'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:34.711042 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-d8dec4d8-c911-4421-9c8e-26a538f75059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73c9cf0f579144eb80c090456d363a05', 'request_id': 'req-d8dec4d8-c911-4421-9c8e-26a538f75059'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:34.716839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-d8dec4d8-c911-4421-9c8e-26a538f75059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73c9cf0f579144eb80c090456d363a05', 'request_id': 'req-d8dec4d8-c911-4421-9c8e-26a538f75059'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:34.728531 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-d8dec4d8-c911-4421-9c8e-26a538f75059 admin admin] The token cache is being invalidate because project 73c9cf0f579144eb80c090456d363a05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:13:34.728843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-d8dec4d8-c911-4421-9c8e-26a538f75059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:34.729579 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-d8dec4d8-c911-4421-9c8e-26a538f75059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:34.731172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1179/2353] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:13:34 2022] DELETE /identity/v3/projects/73c9cf0f579144eb80c090456d363a05 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:34.736398 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d3e64-3c27-4a70-ac8f-c7e43b8904c8 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:34.736894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d3e64-3c27-4a70-ac8f-c7e43b8904c8 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:34.737020 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d3e64-3c27-4a70-ac8f-c7e43b8904c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:34.758204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-711d3e64-3c27-4a70-ac8f-c7e43b8904c8 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:34.759420 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-711d3e64-3c27-4a70-ac8f-c7e43b8904c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:34.818798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:34.833078 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1175/2354] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:13:34.840898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcc35e-2a24-4b69-9020-826b12d269be None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:34.841795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcc35e-2a24-4b69-9020-826b12d269be None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:34.842203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bcc35e-2a24-4b69-9020-826b12d269be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:34.864306 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-85bcc35e-2a24-4b69-9020-826b12d269be None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:34.865639 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-85bcc35e-2a24-4b69-9020-826b12d269be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:34.881807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1180/2355] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:13:34.883072 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d3e64-3c27-4a70-ac8f-c7e43b8904c8 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:34.892844 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d3e64-3c27-4a70-ac8f-c7e43b8904c8 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:34.897357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbcc6e-7d7a-4f85-adaf-8f745fe00e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:34.897959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbcc6e-7d7a-4f85-adaf-8f745fe00e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:34.898463 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbcc6e-7d7a-4f85-adaf-8f745fe00e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:34.898733 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbcc6e-7d7a-4f85-adaf-8f745fe00e99 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:34.899411 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbcc6e-7d7a-4f85-adaf-8f745fe00e99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:13:34.899900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbcc6e-7d7a-4f85-adaf-8f745fe00e99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:34.901506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbcc6e-7d7a-4f85-adaf-8f745fe00e99 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:34.913925 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1176/2356] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:13:34.918470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcc35e-2a24-4b69-9020-826b12d269be None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:34.923014 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:34.929778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bcc35e-2a24-4b69-9020-826b12d269be None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:34.936391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528731af-bb3f-49a4-8b35-518f49273730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:34.937041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528731af-bb3f-49a4-8b35-518f49273730 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:34.937318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528731af-bb3f-49a4-8b35-518f49273730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:34.937589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528731af-bb3f-49a4-8b35-518f49273730 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:34.938792 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528731af-bb3f-49a4-8b35-518f49273730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:34.940782 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528731af-bb3f-49a4-8b35-518f49273730 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:34.969504 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1181/2357] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:13:34 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:13:34.974102 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbcc6e-7d7a-4f85-adaf-8f745fe00e99 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:34.984328 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbcc6e-7d7a-4f85-adaf-8f745fe00e99 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:35.044271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a83d1b-c8ff-412e-9f93-c6aa6d83b501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:35.044969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a83d1b-c8ff-412e-9f93-c6aa6d83b501 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:35.045316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a83d1b-c8ff-412e-9f93-c6aa6d83b501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:35.045651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a83d1b-c8ff-412e-9f93-c6aa6d83b501 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:35.046440 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a83d1b-c8ff-412e-9f93-c6aa6d83b501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:35.048724 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a83d1b-c8ff-412e-9f93-c6aa6d83b501 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:35.079620 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1177/2358] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:13:34 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:13:35.082773 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528731af-bb3f-49a4-8b35-518f49273730 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:35.093896 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528731af-bb3f-49a4-8b35-518f49273730 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:35.098773 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375afb0-9293-411c-8331-d2863bccb5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:35.099545 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375afb0-9293-411c-8331-d2863bccb5bf admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:35.099869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375afb0-9293-411c-8331-d2863bccb5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:35.100291 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375afb0-9293-411c-8331-d2863bccb5bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:35.100934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375afb0-9293-411c-8331-d2863bccb5bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:35.102678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375afb0-9293-411c-8331-d2863bccb5bf admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:35.114403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1182/2359] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:13:35.118738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a83d1b-c8ff-412e-9f93-c6aa6d83b501 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:35.133232 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a83d1b-c8ff-412e-9f93-c6aa6d83b501 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:35.138315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14989476-a309-4976-be45-8458b8f224f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:35.138917 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14989476-a309-4976-be45-8458b8f224f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:35.139193 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14989476-a309-4976-be45-8458b8f224f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:35.139492 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14989476-a309-4976-be45-8458b8f224f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:35.140146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14989476-a309-4976-be45-8458b8f224f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:35.141788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14989476-a309-4976-be45-8458b8f224f3 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:35.154634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1178/2360] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:35.159053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375afb0-9293-411c-8331-d2863bccb5bf admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:35.171199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375afb0-9293-411c-8331-d2863bccb5bf admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:35.181576 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5b8b0-22f1-4785-90f3-185268a06f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:35.182376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5b8b0-22f1-4785-90f3-185268a06f3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:35.182719 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5b8b0-22f1-4785-90f3-185268a06f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:35.183063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5b8b0-22f1-4785-90f3-185268a06f3a admin admin] PATH_INFO: `/v3/projects/6f66d52c520347d4bd16b4661e7ce44e/users/84d632256bf743ca81315b0f7a21c092/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:35.207806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5b8b0-22f1-4785-90f3-185268a06f3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f66d52c520347d4bd16b4661e7ce44e, user_id=84d632256bf743ca81315b0f7a21c092, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:35.210829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5b8b0-22f1-4785-90f3-185268a06f3a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:35.239161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1183/2361] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:13:35 2022] PUT /identity/v3/projects/6f66d52c520347d4bd16b4661e7ce44e/users/84d632256bf743ca81315b0f7a21c092/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:35.240938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b5191-4664-4e7c-b25c-8b47b48b0225 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:35.241218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b5191-4664-4e7c-b25c-8b47b48b0225 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:35.241559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b5191-4664-4e7c-b25c-8b47b48b0225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:35.271214 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-042b5191-4664-4e7c-b25c-8b47b48b0225 None None] MFA Rules not processed for user `84d632256bf743ca81315b0f7a21c092`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:35.272473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-042b5191-4664-4e7c-b25c-8b47b48b0225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:35.356248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1179/2362] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:35.357241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5b8b0-22f1-4785-90f3-185268a06f3a admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:35.434903 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5b8b0-22f1-4785-90f3-185268a06f3a admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:35.440391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac6c26-2446-4a73-a224-f000550dede3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:35.441080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac6c26-2446-4a73-a224-f000550dede3 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:35.441353 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac6c26-2446-4a73-a224-f000550dede3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:35.441618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac6c26-2446-4a73-a224-f000550dede3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:35.442261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac6c26-2446-4a73-a224-f000550dede3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:35.444242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac6c26-2446-4a73-a224-f000550dede3 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:35.473508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1184/2363] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:13:35 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:13:35.477699 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b5191-4664-4e7c-b25c-8b47b48b0225 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:35.489583 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b5191-4664-4e7c-b25c-8b47b48b0225 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:35.543217 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:35.567023 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d423c88-eb19-4391-bc69-428f77943e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:35.567598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d423c88-eb19-4391-bc69-428f77943e2a admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:35.567877 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d423c88-eb19-4391-bc69-428f77943e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:35.568147 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d423c88-eb19-4391-bc69-428f77943e2a admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:35.568797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d423c88-eb19-4391-bc69-428f77943e2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:35.571261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d423c88-eb19-4391-bc69-428f77943e2a admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:35.599290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1180/2364] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:13:35 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:13:35.603297 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac6c26-2446-4a73-a224-f000550dede3 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:35.614029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac6c26-2446-4a73-a224-f000550dede3 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:35.619966 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f22f54-bcd1-4e34-98f0-9c82ce40ebc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:35.620730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f22f54-bcd1-4e34-98f0-9c82ce40ebc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:35.621073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f22f54-bcd1-4e34-98f0-9c82ce40ebc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:35.621414 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f22f54-bcd1-4e34-98f0-9c82ce40ebc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:35.622799 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f22f54-bcd1-4e34-98f0-9c82ce40ebc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:35.624691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f22f54-bcd1-4e34-98f0-9c82ce40ebc7 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:35.637052 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1185/2365] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:13:35.641056 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d423c88-eb19-4391-bc69-428f77943e2a admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:35.652744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d423c88-eb19-4391-bc69-428f77943e2a admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:35.663004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f3b2e-df7e-4c31-b94b-07a8843ae378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:35.664961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f3b2e-df7e-4c31-b94b-07a8843ae378 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:35.665390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f3b2e-df7e-4c31-b94b-07a8843ae378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:35.665814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f3b2e-df7e-4c31-b94b-07a8843ae378 admin admin] PATH_INFO: `/v3/projects/4ae370f75d534760807eb028b11942af/users/53ada34d8bd94556a37bfa72da345b74/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:35.691879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f3b2e-df7e-4c31-b94b-07a8843ae378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ae370f75d534760807eb028b11942af, user_id=53ada34d8bd94556a37bfa72da345b74, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:35.694606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9f3b2e-df7e-4c31-b94b-07a8843ae378 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:35.720738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1181/2366] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:13:35 2022] PUT /identity/v3/projects/4ae370f75d534760807eb028b11942af/users/53ada34d8bd94556a37bfa72da345b74/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:35.724663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f22f54-bcd1-4e34-98f0-9c82ce40ebc7 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:35.735303 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f22f54-bcd1-4e34-98f0-9c82ce40ebc7 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:35.826985 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa63cc-7740-4e04-ae49-65f6ee0def8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:35.827794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa63cc-7740-4e04-ae49-65f6ee0def8d admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:35.828171 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa63cc-7740-4e04-ae49-65f6ee0def8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:35.828539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa63cc-7740-4e04-ae49-65f6ee0def8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:35.829365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa63cc-7740-4e04-ae49-65f6ee0def8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:35.831103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aa63cc-7740-4e04-ae49-65f6ee0def8d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:35.846999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f3b2e-df7e-4c31-b94b-07a8843ae378 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:35.851589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1186/2367] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:35.859401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f3b2e-df7e-4c31-b94b-07a8843ae378 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:35.864343 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bc998-7fa3-43dc-b518-83f71ad3240c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:35.865177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bc998-7fa3-43dc-b518-83f71ad3240c admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:35.865520 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bc998-7fa3-43dc-b518-83f71ad3240c admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:35.865862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bc998-7fa3-43dc-b518-83f71ad3240c admin admin] PATH_INFO: `/v3/projects/4ae370f75d534760807eb028b11942af/users/53ada34d8bd94556a37bfa72da345b74/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:35.868183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bc998-7fa3-43dc-b518-83f71ad3240c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ae370f75d534760807eb028b11942af, user_id=53ada34d8bd94556a37bfa72da345b74, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:35.871337 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bc998-7fa3-43dc-b518-83f71ad3240c admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:35.882374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1182/2368] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:13:35 2022] PUT /identity/v3/projects/4ae370f75d534760807eb028b11942af/users/53ada34d8bd94556a37bfa72da345b74/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:13:35.897401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465b875-eb96-44ea-946c-59d11fa24466 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:35.897896 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465b875-eb96-44ea-946c-59d11fa24466 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:35.898690 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4465b875-eb96-44ea-946c-59d11fa24466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:35.940557 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-4465b875-eb96-44ea-946c-59d11fa24466 None None] MFA Rules not processed for user `53ada34d8bd94556a37bfa72da345b74`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:35.942113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4465b875-eb96-44ea-946c-59d11fa24466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:35.947643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:36.055480 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1187/2369] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:36.066247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bc998-7fa3-43dc-b518-83f71ad3240c admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:36.068154 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-448bc998-7fa3-43dc-b518-83f71ad3240c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:36.130547 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bc998-7fa3-43dc-b518-83f71ad3240c admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:36.135039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402adb01-2e9d-45a5-ba29-312866c948b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:36.135699 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402adb01-2e9d-45a5-ba29-312866c948b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:36.135972 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402adb01-2e9d-45a5-ba29-312866c948b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:36.136233 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402adb01-2e9d-45a5-ba29-312866c948b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:36.146076 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402adb01-2e9d-45a5-ba29-312866c948b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:36.147553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402adb01-2e9d-45a5-ba29-312866c948b0 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:36.157668 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1183/2370] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:36.595599 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465b875-eb96-44ea-946c-59d11fa24466 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:36.671829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4465b875-eb96-44ea-946c-59d11fa24466 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:36.676656 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18869fe-6d95-406d-84a9-baadd8d42a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:36.677299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18869fe-6d95-406d-84a9-baadd8d42a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:36.677581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18869fe-6d95-406d-84a9-baadd8d42a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:36.677866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18869fe-6d95-406d-84a9-baadd8d42a5f admin admin] PATH_INFO: `/v3/users/84d632256bf743ca81315b0f7a21c092` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:36.679021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18869fe-6d95-406d-84a9-baadd8d42a5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=84d632256bf743ca81315b0f7a21c092)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:36.681024 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18869fe-6d95-406d-84a9-baadd8d42a5f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:36.731207 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e18869fe-6d95-406d-84a9-baadd8d42a5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84d632256bf743ca81315b0f7a21c092'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:36.738559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e18869fe-6d95-406d-84a9-baadd8d42a5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84d632256bf743ca81315b0f7a21c092'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:36.739198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e18869fe-6d95-406d-84a9-baadd8d42a5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84d632256bf743ca81315b0f7a21c092'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:36.739882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e18869fe-6d95-406d-84a9-baadd8d42a5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84d632256bf743ca81315b0f7a21c092'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:36.746598 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e18869fe-6d95-406d-84a9-baadd8d42a5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84d632256bf743ca81315b0f7a21c092'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:36.764445 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1188/2371] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:13:36 2022] DELETE /identity/v3/users/84d632256bf743ca81315b0f7a21c092 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:36.769210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd4b18-a7c9-4fc4-9ff6-38991404afe8 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:36.769494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd4b18-a7c9-4fc4-9ff6-38991404afe8 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:36.769755 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd4b18-a7c9-4fc4-9ff6-38991404afe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:36.789469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-21bd4b18-a7c9-4fc4-9ff6-38991404afe8 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:36.790690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-21bd4b18-a7c9-4fc4-9ff6-38991404afe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:36.853341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1184/2372] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:36.863601 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18869fe-6d95-406d-84a9-baadd8d42a5f admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:36.865158 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e18869fe-6d95-406d-84a9-baadd8d42a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:36.888710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:36.931344 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18869fe-6d95-406d-84a9-baadd8d42a5f admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:36.935466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59611f-b4c3-4554-bec0-9f358d9ce9cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:36.936126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59611f-b4c3-4554-bec0-9f358d9ce9cc service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:36.936410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59611f-b4c3-4554-bec0-9f358d9ce9cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:36.936669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59611f-b4c3-4554-bec0-9f358d9ce9cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:36.946686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59611f-b4c3-4554-bec0-9f358d9ce9cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:36.948155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a59611f-b4c3-4554-bec0-9f358d9ce9cc service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:36.958961 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1189/2373] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:37.455236 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd4b18-a7c9-4fc4-9ff6-38991404afe8 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:37.457011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-21bd4b18-a7c9-4fc4-9ff6-38991404afe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:37.484390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bd4b18-a7c9-4fc4-9ff6-38991404afe8 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:37.490616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093f68d-f637-464c-959b-851b17c7d1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:37.491504 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093f68d-f637-464c-959b-851b17c7d1ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:37.491838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093f68d-f637-464c-959b-851b17c7d1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:37.492258 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b093f68d-f637-464c-959b-851b17c7d1ff admin admin] PATH_INFO: `/v3/projects/6f66d52c520347d4bd16b4661e7ce44e` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:37.493908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093f68d-f637-464c-959b-851b17c7d1ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f66d52c520347d4bd16b4661e7ce44e)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:37.496064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b093f68d-f637-464c-959b-851b17c7d1ff admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:37.541871 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:37.549315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b093f68d-f637-464c-959b-851b17c7d1ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f66d52c520347d4bd16b4661e7ce44e', 'request_id': 'req-b093f68d-f637-464c-959b-851b17c7d1ff'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:37.558197 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b093f68d-f637-464c-959b-851b17c7d1ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f66d52c520347d4bd16b4661e7ce44e', 'request_id': 'req-b093f68d-f637-464c-959b-851b17c7d1ff'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:37.565148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b093f68d-f637-464c-959b-851b17c7d1ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f66d52c520347d4bd16b4661e7ce44e', 'request_id': 'req-b093f68d-f637-464c-959b-851b17c7d1ff'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:37.577526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b093f68d-f637-464c-959b-851b17c7d1ff admin admin] The token cache is being invalidate because project 6f66d52c520347d4bd16b4661e7ce44e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:13:37.577813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b093f68d-f637-464c-959b-851b17c7d1ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:37.578635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b093f68d-f637-464c-959b-851b17c7d1ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:37.580179 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1185/2374] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:13:37 2022] DELETE /identity/v3/projects/6f66d52c520347d4bd16b4661e7ce44e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:37.584580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59611f-b4c3-4554-bec0-9f358d9ce9cc service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:37.594806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59611f-b4c3-4554-bec0-9f358d9ce9cc service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:37.656446 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b234602-b8d2-41c5-9148-20fcdcb3136c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:37.657192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b234602-b8d2-41c5-9148-20fcdcb3136c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:37.657389 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b234602-b8d2-41c5-9148-20fcdcb3136c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:37.657617 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b234602-b8d2-41c5-9148-20fcdcb3136c admin admin] PATH_INFO: `/v3/users/53ada34d8bd94556a37bfa72da345b74` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:37.658812 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b234602-b8d2-41c5-9148-20fcdcb3136c admin admin] RBAC: Authorizing `identity:delete_user(user_id=53ada34d8bd94556a37bfa72da345b74)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:37.660806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b234602-b8d2-41c5-9148-20fcdcb3136c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:37.710766 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5b234602-b8d2-41c5-9148-20fcdcb3136c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53ada34d8bd94556a37bfa72da345b74'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:37.718651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5b234602-b8d2-41c5-9148-20fcdcb3136c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53ada34d8bd94556a37bfa72da345b74'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:37.719286 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5b234602-b8d2-41c5-9148-20fcdcb3136c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53ada34d8bd94556a37bfa72da345b74'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:37.719983 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5b234602-b8d2-41c5-9148-20fcdcb3136c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53ada34d8bd94556a37bfa72da345b74'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:37.727821 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5b234602-b8d2-41c5-9148-20fcdcb3136c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53ada34d8bd94556a37bfa72da345b74'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:37.747176 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1190/2375] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:13:37 2022] DELETE /identity/v3/users/53ada34d8bd94556a37bfa72da345b74 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:38.029961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093f68d-f637-464c-959b-851b17c7d1ff admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:38.031710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b093f68d-f637-464c-959b-851b17c7d1ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:38.049925 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b093f68d-f637-464c-959b-851b17c7d1ff admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:38.109732 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8863bc48-249e-41e4-82d4-cf0df6b51c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:38.110395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863bc48-249e-41e4-82d4-cf0df6b51c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:38.110697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863bc48-249e-41e4-82d4-cf0df6b51c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:38.110958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863bc48-249e-41e4-82d4-cf0df6b51c58 admin admin] PATH_INFO: `/v3/projects/4ae370f75d534760807eb028b11942af` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:38.112083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863bc48-249e-41e4-82d4-cf0df6b51c58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ae370f75d534760807eb028b11942af)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:38.114000 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8863bc48-249e-41e4-82d4-cf0df6b51c58 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:38.164111 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8863bc48-249e-41e4-82d4-cf0df6b51c58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ae370f75d534760807eb028b11942af', 'request_id': 'req-8863bc48-249e-41e4-82d4-cf0df6b51c58'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:38.175472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8863bc48-249e-41e4-82d4-cf0df6b51c58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ae370f75d534760807eb028b11942af', 'request_id': 'req-8863bc48-249e-41e4-82d4-cf0df6b51c58'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:38.183192 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8863bc48-249e-41e4-82d4-cf0df6b51c58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ae370f75d534760807eb028b11942af', 'request_id': 'req-8863bc48-249e-41e4-82d4-cf0df6b51c58'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:38.196050 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8863bc48-249e-41e4-82d4-cf0df6b51c58 admin admin] The token cache is being invalidate because project 4ae370f75d534760807eb028b11942af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:13:38.196320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8863bc48-249e-41e4-82d4-cf0df6b51c58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:38.196989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8863bc48-249e-41e4-82d4-cf0df6b51c58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:38.198465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1186/2376] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:13:38 2022] DELETE /identity/v3/projects/4ae370f75d534760807eb028b11942af => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:13:38.203390 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a760921-4f70-4597-96bc-8315dd77d8ef None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:38.203719 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a760921-4f70-4597-96bc-8315dd77d8ef None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:38.204036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a760921-4f70-4597-96bc-8315dd77d8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:38.225101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-3a760921-4f70-4597-96bc-8315dd77d8ef None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:38.226919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3a760921-4f70-4597-96bc-8315dd77d8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:38.270739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:38.296888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1191/2377] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:38.305003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44bb2e-2e3e-4a64-922f-adea5059eb80 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:38.305295 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44bb2e-2e3e-4a64-922f-adea5059eb80 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:38.305572 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44bb2e-2e3e-4a64-922f-adea5059eb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:38.326843 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-df44bb2e-2e3e-4a64-922f-adea5059eb80 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:38.328107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-df44bb2e-2e3e-4a64-922f-adea5059eb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:38.337985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1187/2378] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:38.342490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a760921-4f70-4597-96bc-8315dd77d8ef None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:38.352507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a760921-4f70-4597-96bc-8315dd77d8ef None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:38.359333 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4e7c7-7b87-4681-b202-f5d621121c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:38.359984 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4e7c7-7b87-4681-b202-f5d621121c5e admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:38.360251 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4e7c7-7b87-4681-b202-f5d621121c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:38.360520 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f4e7c7-7b87-4681-b202-f5d621121c5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:38.361213 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4e7c7-7b87-4681-b202-f5d621121c5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:13:38.361516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4e7c7-7b87-4681-b202-f5d621121c5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:38.363119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f4e7c7-7b87-4681-b202-f5d621121c5e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:38.375537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1192/2379] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:13:38.380055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44bb2e-2e3e-4a64-922f-adea5059eb80 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:38.389616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44bb2e-2e3e-4a64-922f-adea5059eb80 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:38.394742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82ad64-9cc8-4dec-a3ca-7d877f4aaaf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:38.395361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82ad64-9cc8-4dec-a3ca-7d877f4aaaf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:38.395665 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82ad64-9cc8-4dec-a3ca-7d877f4aaaf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:38.395934 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82ad64-9cc8-4dec-a3ca-7d877f4aaaf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:38.396593 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82ad64-9cc8-4dec-a3ca-7d877f4aaaf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:38.398651 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82ad64-9cc8-4dec-a3ca-7d877f4aaaf0 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:38.430579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1188/2380] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:13:38 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:13:38.431934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4e7c7-7b87-4681-b202-f5d621121c5e admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:38.442963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f4e7c7-7b87-4681-b202-f5d621121c5e admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:38.506495 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb7975-841c-476a-a078-87eb6838b24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:38.507137 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb7975-841c-476a-a078-87eb6838b24e admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:38.507565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb7975-841c-476a-a078-87eb6838b24e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:38.507844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecb7975-841c-476a-a078-87eb6838b24e admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:38.508482 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb7975-841c-476a-a078-87eb6838b24e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:38.510849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecb7975-841c-476a-a078-87eb6838b24e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:38.544664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1193/2381] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:13:38 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:13:38.548745 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82ad64-9cc8-4dec-a3ca-7d877f4aaaf0 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:38.559740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82ad64-9cc8-4dec-a3ca-7d877f4aaaf0 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:38.564779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0a1ee-94b6-4bad-bccb-a3e54c13e2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:38.565398 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0a1ee-94b6-4bad-bccb-a3e54c13e2cc admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:38.565671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0a1ee-94b6-4bad-bccb-a3e54c13e2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:38.565936 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f0a1ee-94b6-4bad-bccb-a3e54c13e2cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:38.566626 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0a1ee-94b6-4bad-bccb-a3e54c13e2cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:38.568266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f0a1ee-94b6-4bad-bccb-a3e54c13e2cc admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:38.580026 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1189/2382] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:38.584138 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb7975-841c-476a-a078-87eb6838b24e admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:38.594687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecb7975-841c-476a-a078-87eb6838b24e admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:38.599415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef8460-16e3-4851-89fb-7591c9262440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:38.600065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef8460-16e3-4851-89fb-7591c9262440 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:38.600343 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef8460-16e3-4851-89fb-7591c9262440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:38.600616 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef8460-16e3-4851-89fb-7591c9262440 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:38.601268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef8460-16e3-4851-89fb-7591c9262440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:38.602972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef8460-16e3-4851-89fb-7591c9262440 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:38.621710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1194/2383] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:38.625639 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0a1ee-94b6-4bad-bccb-a3e54c13e2cc admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:38.637189 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f0a1ee-94b6-4bad-bccb-a3e54c13e2cc admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:38.642055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad07eeb-49a8-492b-9a0c-00067439ce8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:38.642775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad07eeb-49a8-492b-9a0c-00067439ce8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:38.643055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad07eeb-49a8-492b-9a0c-00067439ce8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:38.643337 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad07eeb-49a8-492b-9a0c-00067439ce8e admin admin] PATH_INFO: `/v3/projects/406ad0e299b54899929bd66245a4c8f0/users/97fef6e276af4a6688475e49da201146/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:38.659954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad07eeb-49a8-492b-9a0c-00067439ce8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=406ad0e299b54899929bd66245a4c8f0, user_id=97fef6e276af4a6688475e49da201146, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:38.662779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad07eeb-49a8-492b-9a0c-00067439ce8e admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:38.666263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:38.696043 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1190/2384] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:13:38 2022] PUT /identity/v3/projects/406ad0e299b54899929bd66245a4c8f0/users/97fef6e276af4a6688475e49da201146/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:13:38.707847 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275deb0-6c4f-4a2c-bb2b-9c5b596fb3ab None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:38.708327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275deb0-6c4f-4a2c-bb2b-9c5b596fb3ab None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:38.708461 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275deb0-6c4f-4a2c-bb2b-9c5b596fb3ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:38.736649 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-e275deb0-6c4f-4a2c-bb2b-9c5b596fb3ab None None] MFA Rules not processed for user `97fef6e276af4a6688475e49da201146`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:38.737876 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e275deb0-6c4f-4a2c-bb2b-9c5b596fb3ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:38.810674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1195/2385] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:38.815712 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad07eeb-49a8-492b-9a0c-00067439ce8e admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:38.895534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad07eeb-49a8-492b-9a0c-00067439ce8e admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:38.901552 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619301a-e22c-403c-9301-49ca5b1d80e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:38.902171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619301a-e22c-403c-9301-49ca5b1d80e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:38.902891 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619301a-e22c-403c-9301-49ca5b1d80e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:38.902891 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d619301a-e22c-403c-9301-49ca5b1d80e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:38.903348 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619301a-e22c-403c-9301-49ca5b1d80e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:38.905434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d619301a-e22c-403c-9301-49ca5b1d80e1 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:38.962854 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1191/2386] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:13:38 2022] POST /identity/v3/projects => generated 369 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:13:38.967203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275deb0-6c4f-4a2c-bb2b-9c5b596fb3ab None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:38.981972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275deb0-6c4f-4a2c-bb2b-9c5b596fb3ab None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:39.024752 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:39.057960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddd843-678e-46cf-ad51-2a679e7184b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:39.060219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddd843-678e-46cf-ad51-2a679e7184b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:39.060510 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddd843-678e-46cf-ad51-2a679e7184b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:39.060904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ddd843-678e-46cf-ad51-2a679e7184b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:39.061706 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ddd843-678e-46cf-ad51-2a679e7184b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:39.064675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ddd843-678e-46cf-ad51-2a679e7184b0 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:39.094167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1196/2387] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:13:38 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:13:39.098633 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619301a-e22c-403c-9301-49ca5b1d80e1 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:39.127868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d619301a-e22c-403c-9301-49ca5b1d80e1 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:39.134297 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988cf34-e571-4413-a3ee-2a49e61e53a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:39.134951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988cf34-e571-4413-a3ee-2a49e61e53a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:39.135230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988cf34-e571-4413-a3ee-2a49e61e53a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:39.135543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988cf34-e571-4413-a3ee-2a49e61e53a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:39.136412 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988cf34-e571-4413-a3ee-2a49e61e53a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:39.138038 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988cf34-e571-4413-a3ee-2a49e61e53a4 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:39.157674 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1192/2388] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:39.158892 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddd843-678e-46cf-ad51-2a679e7184b0 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:39.175206 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ddd843-678e-46cf-ad51-2a679e7184b0 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:39.185183 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb4bc6-fa0c-4777-bdae-ad464988a583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:39.186096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb4bc6-fa0c-4777-bdae-ad464988a583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:39.186462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb4bc6-fa0c-4777-bdae-ad464988a583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:39.186829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb4bc6-fa0c-4777-bdae-ad464988a583 admin admin] PATH_INFO: `/v3/projects/931ad1e963014d16bf7799e73a072b16/users/7879dd1235e24a9a820403e2d5d556e9/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:39.204649 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb4bc6-fa0c-4777-bdae-ad464988a583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=931ad1e963014d16bf7799e73a072b16, user_id=7879dd1235e24a9a820403e2d5d556e9, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:39.207398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cb4bc6-fa0c-4777-bdae-ad464988a583 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:39.222751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1197/2389] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:13:39 2022] PUT /identity/v3/projects/931ad1e963014d16bf7799e73a072b16/users/7879dd1235e24a9a820403e2d5d556e9/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:39.226831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988cf34-e571-4413-a3ee-2a49e61e53a4 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:39.238020 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988cf34-e571-4413-a3ee-2a49e61e53a4 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:39.297184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5886903c-0d68-47cc-858f-a76e5ea7d44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:39.297892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886903c-0d68-47cc-858f-a76e5ea7d44a admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:39.298291 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886903c-0d68-47cc-858f-a76e5ea7d44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:39.298624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5886903c-0d68-47cc-858f-a76e5ea7d44a admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:39.299421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886903c-0d68-47cc-858f-a76e5ea7d44a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:39.301262 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5886903c-0d68-47cc-858f-a76e5ea7d44a admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:39.316287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1193/2390] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:13:39.317902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb4bc6-fa0c-4777-bdae-ad464988a583 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:39.329815 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb4bc6-fa0c-4777-bdae-ad464988a583 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:39.335587 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe4ccc-2b8f-4b90-b613-58c3041c8c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:39.336347 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe4ccc-2b8f-4b90-b613-58c3041c8c27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:39.336775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe4ccc-2b8f-4b90-b613-58c3041c8c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:39.337053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe4ccc-2b8f-4b90-b613-58c3041c8c27 admin admin] PATH_INFO: `/v3/projects/931ad1e963014d16bf7799e73a072b16/users/7879dd1235e24a9a820403e2d5d556e9/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:39.339395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe4ccc-2b8f-4b90-b613-58c3041c8c27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=931ad1e963014d16bf7799e73a072b16, user_id=7879dd1235e24a9a820403e2d5d556e9, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:39.341816 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe4ccc-2b8f-4b90-b613-58c3041c8c27 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:39.354159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1198/2391] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:13:39 2022] PUT /identity/v3/projects/931ad1e963014d16bf7799e73a072b16/users/7879dd1235e24a9a820403e2d5d556e9/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:39.364609 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a23253-3fd4-4d1b-908b-46d71796862f None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:39.364923 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a23253-3fd4-4d1b-908b-46d71796862f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:39.365165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a23253-3fd4-4d1b-908b-46d71796862f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:39.395590 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-d6a23253-3fd4-4d1b-908b-46d71796862f None None] MFA Rules not processed for user `7879dd1235e24a9a820403e2d5d556e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:39.396930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d6a23253-3fd4-4d1b-908b-46d71796862f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:39.460288 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:39.478526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1194/2392] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:39.489903 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe4ccc-2b8f-4b90-b613-58c3041c8c27 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:39.491813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1cfe4ccc-2b8f-4b90-b613-58c3041c8c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:39.550161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe4ccc-2b8f-4b90-b613-58c3041c8c27 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:39.554262 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba8a37-f1c7-48b6-aaa1-f234e7354ff3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:39.554921 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba8a37-f1c7-48b6-aaa1-f234e7354ff3 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:39.555180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba8a37-f1c7-48b6-aaa1-f234e7354ff3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:39.555481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba8a37-f1c7-48b6-aaa1-f234e7354ff3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:39.565380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba8a37-f1c7-48b6-aaa1-f234e7354ff3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:39.566901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba8a37-f1c7-48b6-aaa1-f234e7354ff3 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:39.577159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1199/2393] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:41.051324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a23253-3fd4-4d1b-908b-46d71796862f None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:41.121345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a23253-3fd4-4d1b-908b-46d71796862f None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:41.125910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae1438-d1fb-40a8-b707-a37aef104bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:41.126543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae1438-d1fb-40a8-b707-a37aef104bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:41.126816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae1438-d1fb-40a8-b707-a37aef104bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:41.127086 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae1438-d1fb-40a8-b707-a37aef104bb2 admin admin] PATH_INFO: `/v3/users/97fef6e276af4a6688475e49da201146` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:41.128238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae1438-d1fb-40a8-b707-a37aef104bb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97fef6e276af4a6688475e49da201146)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:41.130169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae1438-d1fb-40a8-b707-a37aef104bb2 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:41.187360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-42ae1438-d1fb-40a8-b707-a37aef104bb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97fef6e276af4a6688475e49da201146'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:41.196964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-42ae1438-d1fb-40a8-b707-a37aef104bb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97fef6e276af4a6688475e49da201146'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:41.197802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-42ae1438-d1fb-40a8-b707-a37aef104bb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97fef6e276af4a6688475e49da201146'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:41.198560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-42ae1438-d1fb-40a8-b707-a37aef104bb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97fef6e276af4a6688475e49da201146'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:41.213461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-42ae1438-d1fb-40a8-b707-a37aef104bb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97fef6e276af4a6688475e49da201146'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:41.229918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1195/2394] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:13:41 2022] DELETE /identity/v3/users/97fef6e276af4a6688475e49da201146 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:41.235850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1df0a-e50b-49be-a113-eac42c5cffa9 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:41.236335 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1df0a-e50b-49be-a113-eac42c5cffa9 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:41.236647 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1df0a-e50b-49be-a113-eac42c5cffa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:41.257188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-34b1df0a-e50b-49be-a113-eac42c5cffa9 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:41.258726 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-34b1df0a-e50b-49be-a113-eac42c5cffa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:41.327641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1200/2395] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:41.338052 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae1438-d1fb-40a8-b707-a37aef104bb2 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:41.339470 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-42ae1438-d1fb-40a8-b707-a37aef104bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:41.351340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1df0a-e50b-49be-a113-eac42c5cffa9 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:41.352923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-34b1df0a-e50b-49be-a113-eac42c5cffa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:41.393860 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:41.412362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae1438-d1fb-40a8-b707-a37aef104bb2 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:41.416556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e1e6f-0137-4c8e-b37f-44a654f7d416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:41.417174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e1e6f-0137-4c8e-b37f-44a654f7d416 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:41.417459 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e1e6f-0137-4c8e-b37f-44a654f7d416 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:41.418191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e1e6f-0137-4c8e-b37f-44a654f7d416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:41.434214 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e1e6f-0137-4c8e-b37f-44a654f7d416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:41.436078 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e1e6f-0137-4c8e-b37f-44a654f7d416 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:41.439161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1df0a-e50b-49be-a113-eac42c5cffa9 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:41.444872 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e03524-2c01-4468-93a7-f077ce325568 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:41.445559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e03524-2c01-4468-93a7-f077ce325568 service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:41.445869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e03524-2c01-4468-93a7-f077ce325568 service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:41.446240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e03524-2c01-4468-93a7-f077ce325568 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:41.447994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b9e03524-2c01-4468-93a7-f077ce325568 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:41.448475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1196/2396] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:41.460583 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e03524-2c01-4468-93a7-f077ce325568 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:41.462229 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e03524-2c01-4468-93a7-f077ce325568 service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:41.483596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1201/2397] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:41.732671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e1e6f-0137-4c8e-b37f-44a654f7d416 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:41.734161 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b53e1e6f-0137-4c8e-b37f-44a654f7d416 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:41.749335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e1e6f-0137-4c8e-b37f-44a654f7d416 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:41.755320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:41.756186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:41.756498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:41.756770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa admin admin] PATH_INFO: `/v3/projects/406ad0e299b54899929bd66245a4c8f0` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:41.757915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=406ad0e299b54899929bd66245a4c8f0)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:41.759954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:41.768428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:41.809348 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '406ad0e299b54899929bd66245a4c8f0', 'request_id': 'req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:41.818382 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '406ad0e299b54899929bd66245a4c8f0', 'request_id': 'req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:41.824758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '406ad0e299b54899929bd66245a4c8f0', 'request_id': 'req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:41.836303 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa admin admin] The token cache is being invalidate because project 406ad0e299b54899929bd66245a4c8f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:13:41.836589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:41.837215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:41.838735 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1197/2398] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:13:41 2022] DELETE /identity/v3/projects/406ad0e299b54899929bd66245a4c8f0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:41.842939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e03524-2c01-4468-93a7-f077ce325568 service nova] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:41.853759 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e03524-2c01-4468-93a7-f077ce325568 service nova] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:41.917034 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26b082-1fa1-4f3a-ab57-9d13e5892afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:41.917675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26b082-1fa1-4f3a-ab57-9d13e5892afb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:41.917957 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26b082-1fa1-4f3a-ab57-9d13e5892afb admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:41.918237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26b082-1fa1-4f3a-ab57-9d13e5892afb admin admin] PATH_INFO: `/v3/users/7879dd1235e24a9a820403e2d5d556e9` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:41.919428 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26b082-1fa1-4f3a-ab57-9d13e5892afb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7879dd1235e24a9a820403e2d5d556e9)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:41.921404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26b082-1fa1-4f3a-ab57-9d13e5892afb admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:41.974076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-ab26b082-1fa1-4f3a-ab57-9d13e5892afb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7879dd1235e24a9a820403e2d5d556e9'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:41.981664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-ab26b082-1fa1-4f3a-ab57-9d13e5892afb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7879dd1235e24a9a820403e2d5d556e9'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:41.982292 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-ab26b082-1fa1-4f3a-ab57-9d13e5892afb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7879dd1235e24a9a820403e2d5d556e9'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:41.982955 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-ab26b082-1fa1-4f3a-ab57-9d13e5892afb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7879dd1235e24a9a820403e2d5d556e9'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:41.989882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-ab26b082-1fa1-4f3a-ab57-9d13e5892afb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7879dd1235e24a9a820403e2d5d556e9'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:42.008557 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1202/2399] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:13:41 2022] DELETE /identity/v3/users/7879dd1235e24a9a820403e2d5d556e9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:42.312110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:42.315295 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:42.332143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0cc2b-9997-4a61-8ec9-1f72e036e6fa admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:42.406845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd03ba-96e9-4301-b626-b573ba6526d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:42.407556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd03ba-96e9-4301-b626-b573ba6526d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:42.407832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd03ba-96e9-4301-b626-b573ba6526d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:42.408138 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd03ba-96e9-4301-b626-b573ba6526d9 admin admin] PATH_INFO: `/v3/projects/931ad1e963014d16bf7799e73a072b16` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:42.409309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd03ba-96e9-4301-b626-b573ba6526d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=931ad1e963014d16bf7799e73a072b16)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:42.411224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdd03ba-96e9-4301-b626-b573ba6526d9 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:42.462029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1cdd03ba-96e9-4301-b626-b573ba6526d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '931ad1e963014d16bf7799e73a072b16', 'request_id': 'req-1cdd03ba-96e9-4301-b626-b573ba6526d9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:42.471402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1cdd03ba-96e9-4301-b626-b573ba6526d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '931ad1e963014d16bf7799e73a072b16', 'request_id': 'req-1cdd03ba-96e9-4301-b626-b573ba6526d9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:42.477487 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1cdd03ba-96e9-4301-b626-b573ba6526d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '931ad1e963014d16bf7799e73a072b16', 'request_id': 'req-1cdd03ba-96e9-4301-b626-b573ba6526d9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:42.488925 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1cdd03ba-96e9-4301-b626-b573ba6526d9 admin admin] The token cache is being invalidate because project 931ad1e963014d16bf7799e73a072b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:13:42.489216 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1cdd03ba-96e9-4301-b626-b573ba6526d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:42.489880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1cdd03ba-96e9-4301-b626-b573ba6526d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:42.491414 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1198/2400] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:13:42 2022] DELETE /identity/v3/projects/931ad1e963014d16bf7799e73a072b16 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:42.496591 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f35c4-a66b-4cdf-861f-3c1df73b36c1 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:42.497053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f35c4-a66b-4cdf-861f-3c1df73b36c1 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:42.497175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1f35c4-a66b-4cdf-861f-3c1df73b36c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:42.517174 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-5b1f35c4-a66b-4cdf-861f-3c1df73b36c1 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:42.518377 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5b1f35c4-a66b-4cdf-861f-3c1df73b36c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:42.551372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:42.580365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1203/2401] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:42.588489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae3d49-cf60-4b80-a972-87cbfb8228a3 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:42.588859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae3d49-cf60-4b80-a972-87cbfb8228a3 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:42.589130 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae3d49-cf60-4b80-a972-87cbfb8228a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:42.619741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-beae3d49-cf60-4b80-a972-87cbfb8228a3 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:42.621688 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-beae3d49-cf60-4b80-a972-87cbfb8228a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:42.629970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1199/2402] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:13:42.634364 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f35c4-a66b-4cdf-861f-3c1df73b36c1 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:42.644061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1f35c4-a66b-4cdf-861f-3c1df73b36c1 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:42.648847 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce9934-24eb-4dea-a175-af8f7d6f247c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:42.649457 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce9934-24eb-4dea-a175-af8f7d6f247c admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:42.649731 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce9934-24eb-4dea-a175-af8f7d6f247c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:42.650002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ce9934-24eb-4dea-a175-af8f7d6f247c admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:42.650656 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce9934-24eb-4dea-a175-af8f7d6f247c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:13:42.650955 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce9934-24eb-4dea-a175-af8f7d6f247c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:42.652591 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ce9934-24eb-4dea-a175-af8f7d6f247c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:42.664840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1204/2403] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:13:42.669036 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae3d49-cf60-4b80-a972-87cbfb8228a3 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:42.678344 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae3d49-cf60-4b80-a972-87cbfb8228a3 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:42.683431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b55f20-6ae5-4b85-9fe2-41d953099455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:42.684065 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b55f20-6ae5-4b85-9fe2-41d953099455 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:42.684340 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b55f20-6ae5-4b85-9fe2-41d953099455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:42.684612 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b55f20-6ae5-4b85-9fe2-41d953099455 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:42.685255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b55f20-6ae5-4b85-9fe2-41d953099455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:42.687168 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b55f20-6ae5-4b85-9fe2-41d953099455 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:42.716404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:42.724820 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1200/2404] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:13:42 2022] POST /identity/v3/projects => generated 391 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:13:42.729373 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce9934-24eb-4dea-a175-af8f7d6f247c admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:42.739401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ce9934-24eb-4dea-a175-af8f7d6f247c admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:42.797669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c35cc-d230-4dec-ade7-7d2421b625c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:42.798275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c35cc-d230-4dec-ade7-7d2421b625c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:42.798553 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c35cc-d230-4dec-ade7-7d2421b625c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:42.798821 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144c35cc-d230-4dec-ade7-7d2421b625c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:42.799489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c35cc-d230-4dec-ade7-7d2421b625c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:42.801382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144c35cc-d230-4dec-ade7-7d2421b625c3 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:42.825331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1205/2405] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:13:42 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:13:42.829845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b55f20-6ae5-4b85-9fe2-41d953099455 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:42.841020 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b55f20-6ae5-4b85-9fe2-41d953099455 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:42.846641 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975da14-5315-499c-b9c0-abdd095920b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:42.847246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975da14-5315-499c-b9c0-abdd095920b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:42.847560 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975da14-5315-499c-b9c0-abdd095920b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:42.847824 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975da14-5315-499c-b9c0-abdd095920b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:42.848471 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c975da14-5315-499c-b9c0-abdd095920b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:42.850084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c975da14-5315-499c-b9c0-abdd095920b8 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:42.860723 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1201/2406] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:42.865037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c35cc-d230-4dec-ade7-7d2421b625c3 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:42.876697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144c35cc-d230-4dec-ade7-7d2421b625c3 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:42.882340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdfb2f-30f8-415c-b0ff-78a3c175a73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:42.883010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdfb2f-30f8-415c-b0ff-78a3c175a73f admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:42.883282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdfb2f-30f8-415c-b0ff-78a3c175a73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:42.883587 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdfb2f-30f8-415c-b0ff-78a3c175a73f admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:42.884241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdfb2f-30f8-415c-b0ff-78a3c175a73f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:42.885822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdfb2f-30f8-415c-b0ff-78a3c175a73f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:42.897994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1206/2407] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:42.903054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975da14-5315-499c-b9c0-abdd095920b8 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:42.913957 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975da14-5315-499c-b9c0-abdd095920b8 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:42.918989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1132c-47c1-4a81-9c30-192e8f8fb3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:42.919671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1132c-47c1-4a81-9c30-192e8f8fb3af admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:42.919990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1132c-47c1-4a81-9c30-192e8f8fb3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:42.920208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f1132c-47c1-4a81-9c30-192e8f8fb3af admin admin] PATH_INFO: `/v3/projects/514247d120ac407c8ed1fb1e9cfac103/users/ef40fe46e89f4e50b4500dca85755d95/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:42.937472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1132c-47c1-4a81-9c30-192e8f8fb3af admin admin] RBAC: Authorizing `identity:create_grant(project_id=514247d120ac407c8ed1fb1e9cfac103, user_id=ef40fe46e89f4e50b4500dca85755d95, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:42.940805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f1132c-47c1-4a81-9c30-192e8f8fb3af admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:42.958435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1202/2408] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:13:42 2022] PUT /identity/v3/projects/514247d120ac407c8ed1fb1e9cfac103/users/ef40fe46e89f4e50b4500dca85755d95/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:42.969883 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78683684-f96a-46b3-b4e0-81336399e46b None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:42.970168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78683684-f96a-46b3-b4e0-81336399e46b None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:42.970440 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78683684-f96a-46b3-b4e0-81336399e46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:42.998376 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-78683684-f96a-46b3-b4e0-81336399e46b None None] MFA Rules not processed for user `ef40fe46e89f4e50b4500dca85755d95`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:42.999628 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-78683684-f96a-46b3-b4e0-81336399e46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:43.065208 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1207/2409] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:43.070124 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1132c-47c1-4a81-9c30-192e8f8fb3af admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:43.132163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f1132c-47c1-4a81-9c30-192e8f8fb3af admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:43.136904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803caeb-3cf9-4b11-9ca3-9e9173b77977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:43.137516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803caeb-3cf9-4b11-9ca3-9e9173b77977 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:43.137777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803caeb-3cf9-4b11-9ca3-9e9173b77977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:43.138043 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803caeb-3cf9-4b11-9ca3-9e9173b77977 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:43.138694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803caeb-3cf9-4b11-9ca3-9e9173b77977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:43.140631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803caeb-3cf9-4b11-9ca3-9e9173b77977 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:43.172162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1203/2410] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:13:43 2022] POST /identity/v3/projects => generated 391 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:13:43.176473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78683684-f96a-46b3-b4e0-81336399e46b None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:43.189569 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78683684-f96a-46b3-b4e0-81336399e46b None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:43.220268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:43.252648 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c904f28-1d66-41ff-a7e7-9d5f8e02fb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:43.253183 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c904f28-1d66-41ff-a7e7-9d5f8e02fb35 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:43.253454 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c904f28-1d66-41ff-a7e7-9d5f8e02fb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:43.253792 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c904f28-1d66-41ff-a7e7-9d5f8e02fb35 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:43.254322 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c904f28-1d66-41ff-a7e7-9d5f8e02fb35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:43.256642 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c904f28-1d66-41ff-a7e7-9d5f8e02fb35 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:43.283874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1208/2411] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:13:43 2022] POST /identity/v3/users => generated 302 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:13:43.285252 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803caeb-3cf9-4b11-9ca3-9e9173b77977 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:43.299180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803caeb-3cf9-4b11-9ca3-9e9173b77977 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:43.306973 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e2cbb-2ecb-48a4-9570-601a7c92657c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:43.307621 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e2cbb-2ecb-48a4-9570-601a7c92657c admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:43.307897 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e2cbb-2ecb-48a4-9570-601a7c92657c admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:43.308171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e2cbb-2ecb-48a4-9570-601a7c92657c admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:43.308809 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e2cbb-2ecb-48a4-9570-601a7c92657c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:43.310410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e2cbb-2ecb-48a4-9570-601a7c92657c admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:43.322846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1204/2412] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:43.327198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c904f28-1d66-41ff-a7e7-9d5f8e02fb35 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:43.339352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c904f28-1d66-41ff-a7e7-9d5f8e02fb35 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:43.344821 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c1e95-5af2-4d20-acd9-f9dcfbce3e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:43.345484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c1e95-5af2-4d20-acd9-f9dcfbce3e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:43.345756 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c1e95-5af2-4d20-acd9-f9dcfbce3e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:43.346022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c1e95-5af2-4d20-acd9-f9dcfbce3e22 admin admin] PATH_INFO: `/v3/projects/8500198566184471a705d1b12693ca7b/users/5aeeea4fb6934da69cea7820760d542a/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:43.361473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c1e95-5af2-4d20-acd9-f9dcfbce3e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8500198566184471a705d1b12693ca7b, user_id=5aeeea4fb6934da69cea7820760d542a, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:43.364152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c1e95-5af2-4d20-acd9-f9dcfbce3e22 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:43.377745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1209/2413] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:13:43 2022] PUT /identity/v3/projects/8500198566184471a705d1b12693ca7b/users/5aeeea4fb6934da69cea7820760d542a/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:43.381783 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e2cbb-2ecb-48a4-9570-601a7c92657c admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:43.391740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e2cbb-2ecb-48a4-9570-601a7c92657c admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:43.449074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a8899c-1c60-46e0-8b33-efe6e62833d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:43.449685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8899c-1c60-46e0-8b33-efe6e62833d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:43.449956 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8899c-1c60-46e0-8b33-efe6e62833d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:43.450220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a8899c-1c60-46e0-8b33-efe6e62833d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:43.450860 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8899c-1c60-46e0-8b33-efe6e62833d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:43.452489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a8899c-1c60-46e0-8b33-efe6e62833d8 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:43.462346 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1205/2414] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:43.466608 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c1e95-5af2-4d20-acd9-f9dcfbce3e22 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:43.477573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c1e95-5af2-4d20-acd9-f9dcfbce3e22 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:43.483209 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a878f-6d4b-41f6-86ce-b3c8613bef50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:43.483936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a878f-6d4b-41f6-86ce-b3c8613bef50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:43.484184 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a878f-6d4b-41f6-86ce-b3c8613bef50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:43.484454 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a878f-6d4b-41f6-86ce-b3c8613bef50 admin admin] PATH_INFO: `/v3/projects/8500198566184471a705d1b12693ca7b/users/5aeeea4fb6934da69cea7820760d542a/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:43.486462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a878f-6d4b-41f6-86ce-b3c8613bef50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8500198566184471a705d1b12693ca7b, user_id=5aeeea4fb6934da69cea7820760d542a, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:43.489173 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a878f-6d4b-41f6-86ce-b3c8613bef50 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:43.503149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1210/2415] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:13:43 2022] PUT /identity/v3/projects/8500198566184471a705d1b12693ca7b/users/5aeeea4fb6934da69cea7820760d542a/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:13:43.512985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a759ec7-8184-4877-a084-9a8cf87328c1 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:43.513266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a759ec7-8184-4877-a084-9a8cf87328c1 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:43.513539 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a759ec7-8184-4877-a084-9a8cf87328c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:43.542362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-2a759ec7-8184-4877-a084-9a8cf87328c1 None None] MFA Rules not processed for user `5aeeea4fb6934da69cea7820760d542a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:43.545083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2a759ec7-8184-4877-a084-9a8cf87328c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:43.580888 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:43.648465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1206/2416] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:43 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:43.659358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a878f-6d4b-41f6-86ce-b3c8613bef50 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:43.661323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-db6a878f-6d4b-41f6-86ce-b3c8613bef50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:43.721069 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a878f-6d4b-41f6-86ce-b3c8613bef50 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:43.724708 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfecde70-3db4-4760-b17b-db94e0240279 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:43.725323 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecde70-3db4-4760-b17b-db94e0240279 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:43.725597 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecde70-3db4-4760-b17b-db94e0240279 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:43.725909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecde70-3db4-4760-b17b-db94e0240279 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:43.735461 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfecde70-3db4-4760-b17b-db94e0240279 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:43.736951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfecde70-3db4-4760-b17b-db94e0240279 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:43.747670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1211/2417] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:43 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:43.800331 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a759ec7-8184-4877-a084-9a8cf87328c1 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:43.801734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2a759ec7-8184-4877-a084-9a8cf87328c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:43.826319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a759ec7-8184-4877-a084-9a8cf87328c1 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:43.832144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271343fe-7ab7-466b-ad63-e459ac8094ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:43.832774 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271343fe-7ab7-466b-ad63-e459ac8094ef service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:43.833049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271343fe-7ab7-466b-ad63-e459ac8094ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:43.833322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271343fe-7ab7-466b-ad63-e459ac8094ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:43.919963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271343fe-7ab7-466b-ad63-e459ac8094ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:43.922949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271343fe-7ab7-466b-ad63-e459ac8094ef service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:43.956331 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1207/2418] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:45.806478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645665eb-e8af-4a40-b14b-2a53fed62836 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:45.806991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645665eb-e8af-4a40-b14b-2a53fed62836 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:45.807437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645665eb-e8af-4a40-b14b-2a53fed62836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:45.829762 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-645665eb-e8af-4a40-b14b-2a53fed62836 None None] MFA Rules not processed for user `77aeaed3d55945e49b0eb6e3c0519294`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:45.831053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-645665eb-e8af-4a40-b14b-2a53fed62836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:45.896825 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271343fe-7ab7-466b-ad63-e459ac8094ef service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:45.898451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-271343fe-7ab7-466b-ad63-e459ac8094ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:45.921517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1212/2419] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:46.008716 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271343fe-7ab7-466b-ad63-e459ac8094ef service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:46.021348 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c131018-9f43-4844-9f13-237662f3d300 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:46.021976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c131018-9f43-4844-9f13-237662f3d300 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:46.022256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c131018-9f43-4844-9f13-237662f3d300 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:46.022527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c131018-9f43-4844-9f13-237662f3d300 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:46.024475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-7c131018-9f43-4844-9f13-237662f3d300 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:46.037159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c131018-9f43-4844-9f13-237662f3d300 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:46.038836 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c131018-9f43-4844-9f13-237662f3d300 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:46.065250 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1208/2420] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:47.488336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645665eb-e8af-4a40-b14b-2a53fed62836 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:47.525991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:47.562977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645665eb-e8af-4a40-b14b-2a53fed62836 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:47.567476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e662f5c-547d-4358-9f76-0635eb7c3af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:47.568484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e662f5c-547d-4358-9f76-0635eb7c3af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:47.568484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e662f5c-547d-4358-9f76-0635eb7c3af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:47.568713 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e662f5c-547d-4358-9f76-0635eb7c3af0 admin admin] PATH_INFO: `/v3/users/ef40fe46e89f4e50b4500dca85755d95` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:47.569737 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e662f5c-547d-4358-9f76-0635eb7c3af0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef40fe46e89f4e50b4500dca85755d95)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:47.571841 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e662f5c-547d-4358-9f76-0635eb7c3af0 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:47.630832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-0e662f5c-547d-4358-9f76-0635eb7c3af0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef40fe46e89f4e50b4500dca85755d95'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:47.640145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-0e662f5c-547d-4358-9f76-0635eb7c3af0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef40fe46e89f4e50b4500dca85755d95'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:47.640891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-0e662f5c-547d-4358-9f76-0635eb7c3af0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef40fe46e89f4e50b4500dca85755d95'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:47.641529 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-0e662f5c-547d-4358-9f76-0635eb7c3af0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef40fe46e89f4e50b4500dca85755d95'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:47.650454 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-0e662f5c-547d-4358-9f76-0635eb7c3af0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef40fe46e89f4e50b4500dca85755d95'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:47.669354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1213/2421] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:13:47 2022] DELETE /identity/v3/users/ef40fe46e89f4e50b4500dca85755d95 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:47.679225 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd8b41-2563-4665-a363-09cdc2b99888 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:47.679873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd8b41-2563-4665-a363-09cdc2b99888 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:47.680201 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd8b41-2563-4665-a363-09cdc2b99888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:47.701924 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-92bd8b41-2563-4665-a363-09cdc2b99888 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:47.703260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-92bd8b41-2563-4665-a363-09cdc2b99888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:47.752271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:47.775845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1209/2422] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:13:47.785831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e662f5c-547d-4358-9f76-0635eb7c3af0 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:47.787412 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0e662f5c-547d-4358-9f76-0635eb7c3af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:47.848796 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e662f5c-547d-4358-9f76-0635eb7c3af0 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:47.853651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47869af2-7629-46cf-9fdd-7a83be3b9956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:47.854266 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47869af2-7629-46cf-9fdd-7a83be3b9956 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:47.854538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47869af2-7629-46cf-9fdd-7a83be3b9956 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:47.854802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47869af2-7629-46cf-9fdd-7a83be3b9956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:47.865428 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47869af2-7629-46cf-9fdd-7a83be3b9956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:47.867732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47869af2-7629-46cf-9fdd-7a83be3b9956 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:47.878047 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1214/2423] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:48.150861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd8b41-2563-4665-a363-09cdc2b99888 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:48.152373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-92bd8b41-2563-4665-a363-09cdc2b99888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:48.166772 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd8b41-2563-4665-a363-09cdc2b99888 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:48.171960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8fa43b-728e-482e-bdbf-9bd692e08d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:48.172654 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fa43b-728e-482e-bdbf-9bd692e08d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:48.172908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fa43b-728e-482e-bdbf-9bd692e08d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:48.173174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8fa43b-728e-482e-bdbf-9bd692e08d50 admin admin] PATH_INFO: `/v3/projects/514247d120ac407c8ed1fb1e9cfac103` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:48.174333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fa43b-728e-482e-bdbf-9bd692e08d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=514247d120ac407c8ed1fb1e9cfac103)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:48.176427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8fa43b-728e-482e-bdbf-9bd692e08d50 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:48.230696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ac8fa43b-728e-482e-bdbf-9bd692e08d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '514247d120ac407c8ed1fb1e9cfac103', 'request_id': 'req-ac8fa43b-728e-482e-bdbf-9bd692e08d50'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:48.239246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ac8fa43b-728e-482e-bdbf-9bd692e08d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '514247d120ac407c8ed1fb1e9cfac103', 'request_id': 'req-ac8fa43b-728e-482e-bdbf-9bd692e08d50'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:48.245095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ac8fa43b-728e-482e-bdbf-9bd692e08d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '514247d120ac407c8ed1fb1e9cfac103', 'request_id': 'req-ac8fa43b-728e-482e-bdbf-9bd692e08d50'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:48.257108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ac8fa43b-728e-482e-bdbf-9bd692e08d50 admin admin] The token cache is being invalidate because project 514247d120ac407c8ed1fb1e9cfac103 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:13:48.257418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ac8fa43b-728e-482e-bdbf-9bd692e08d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:48.258171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ac8fa43b-728e-482e-bdbf-9bd692e08d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:48.259852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1210/2424] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:13:48 2022] DELETE /identity/v3/projects/514247d120ac407c8ed1fb1e9cfac103 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:48.263990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47869af2-7629-46cf-9fdd-7a83be3b9956 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:48.274408 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47869af2-7629-46cf-9fdd-7a83be3b9956 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:48.331805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:48.339813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe22ec-a6c5-4978-a160-1ffb35b27d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:48.340728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe22ec-a6c5-4978-a160-1ffb35b27d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:48.341040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe22ec-a6c5-4978-a160-1ffb35b27d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:48.341309 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe22ec-a6c5-4978-a160-1ffb35b27d52 admin admin] PATH_INFO: `/v3/users/5aeeea4fb6934da69cea7820760d542a` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:48.342943 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe22ec-a6c5-4978-a160-1ffb35b27d52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aeeea4fb6934da69cea7820760d542a)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:48.345963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe22ec-a6c5-4978-a160-1ffb35b27d52 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:48.407062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-0efe22ec-a6c5-4978-a160-1ffb35b27d52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aeeea4fb6934da69cea7820760d542a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:48.419618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-0efe22ec-a6c5-4978-a160-1ffb35b27d52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aeeea4fb6934da69cea7820760d542a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:48.420529 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-0efe22ec-a6c5-4978-a160-1ffb35b27d52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aeeea4fb6934da69cea7820760d542a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:48.421828 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-0efe22ec-a6c5-4978-a160-1ffb35b27d52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aeeea4fb6934da69cea7820760d542a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:48.430615 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-0efe22ec-a6c5-4978-a160-1ffb35b27d52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aeeea4fb6934da69cea7820760d542a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:48.452969 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1215/2425] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:13:48 2022] DELETE /identity/v3/users/5aeeea4fb6934da69cea7820760d542a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:48.885592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8fa43b-728e-482e-bdbf-9bd692e08d50 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:48.887083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ac8fa43b-728e-482e-bdbf-9bd692e08d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:48.904817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8fa43b-728e-482e-bdbf-9bd692e08d50 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:48.974659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe2661-ba02-4f8e-829d-2bdbaf93fb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:48.975388 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe2661-ba02-4f8e-829d-2bdbaf93fb7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:48.975753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe2661-ba02-4f8e-829d-2bdbaf93fb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:48.976274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe2661-ba02-4f8e-829d-2bdbaf93fb7e admin admin] PATH_INFO: `/v3/projects/8500198566184471a705d1b12693ca7b` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:48.977620 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe2661-ba02-4f8e-829d-2bdbaf93fb7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8500198566184471a705d1b12693ca7b)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:48.979853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe2661-ba02-4f8e-829d-2bdbaf93fb7e admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:49.046577 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b1fe2661-ba02-4f8e-829d-2bdbaf93fb7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8500198566184471a705d1b12693ca7b', 'request_id': 'req-b1fe2661-ba02-4f8e-829d-2bdbaf93fb7e'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:49.054691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b1fe2661-ba02-4f8e-829d-2bdbaf93fb7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8500198566184471a705d1b12693ca7b', 'request_id': 'req-b1fe2661-ba02-4f8e-829d-2bdbaf93fb7e'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:49.061138 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b1fe2661-ba02-4f8e-829d-2bdbaf93fb7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8500198566184471a705d1b12693ca7b', 'request_id': 'req-b1fe2661-ba02-4f8e-829d-2bdbaf93fb7e'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:49.064939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:49.075279 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b1fe2661-ba02-4f8e-829d-2bdbaf93fb7e admin admin] The token cache is being invalidate because project 8500198566184471a705d1b12693ca7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:13:49.076213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b1fe2661-ba02-4f8e-829d-2bdbaf93fb7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:49.077301 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b1fe2661-ba02-4f8e-829d-2bdbaf93fb7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:49.078899 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1211/2426] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:13:48 2022] DELETE /identity/v3/projects/8500198566184471a705d1b12693ca7b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:49.084023 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b2315-d319-4624-a116-b369381604f6 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:49.084901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b2315-d319-4624-a116-b369381604f6 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:49.085542 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b2315-d319-4624-a116-b369381604f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:49.107317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-b36b2315-d319-4624-a116-b369381604f6 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:49.108753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b36b2315-d319-4624-a116-b369381604f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:49.169613 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1216/2427] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:13:49.178377 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8139969-4b5b-474f-8c42-cfbd48f587ed None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:49.178660 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8139969-4b5b-474f-8c42-cfbd48f587ed None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:49.179303 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8139969-4b5b-474f-8c42-cfbd48f587ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:49.199846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-d8139969-4b5b-474f-8c42-cfbd48f587ed None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:49.201084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d8139969-4b5b-474f-8c42-cfbd48f587ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:49.209121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1212/2428] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:49.213412 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b2315-d319-4624-a116-b369381604f6 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:49.223741 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b2315-d319-4624-a116-b369381604f6 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:49.229752 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d66ba-79a8-49d3-9db8-e23543412777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:49.230795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d66ba-79a8-49d3-9db8-e23543412777 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:49.231138 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d66ba-79a8-49d3-9db8-e23543412777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:49.231417 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d66ba-79a8-49d3-9db8-e23543412777 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:49.232185 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d66ba-79a8-49d3-9db8-e23543412777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:13:49.232496 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d66ba-79a8-49d3-9db8-e23543412777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:49.234284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d66ba-79a8-49d3-9db8-e23543412777 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:49.248321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1217/2429] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:13:49.252712 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8139969-4b5b-474f-8c42-cfbd48f587ed None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:49.262908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8139969-4b5b-474f-8c42-cfbd48f587ed None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:49.267553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c6859-8866-40b8-9b00-131de83b8058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:49.268163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c6859-8866-40b8-9b00-131de83b8058 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:49.268443 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c6859-8866-40b8-9b00-131de83b8058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:49.268709 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c6859-8866-40b8-9b00-131de83b8058 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:49.269342 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c6859-8866-40b8-9b00-131de83b8058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:49.271933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c6859-8866-40b8-9b00-131de83b8058 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:49.301732 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1213/2430] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:13:49 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:13:49.306451 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d66ba-79a8-49d3-9db8-e23543412777 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:49.318628 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d66ba-79a8-49d3-9db8-e23543412777 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:49.392429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5bb66-25e0-4f65-ab4d-eb8f6896a0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:49.393079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5bb66-25e0-4f65-ab4d-eb8f6896a0ab admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:49.393352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5bb66-25e0-4f65-ab4d-eb8f6896a0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:49.393620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5bb66-25e0-4f65-ab4d-eb8f6896a0ab admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:49.394284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5bb66-25e0-4f65-ab4d-eb8f6896a0ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:49.396281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5bb66-25e0-4f65-ab4d-eb8f6896a0ab admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:49.411824 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:49.435554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1218/2431] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:13:49 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:13:49.436657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c6859-8866-40b8-9b00-131de83b8058 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:49.448018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c6859-8866-40b8-9b00-131de83b8058 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:49.452972 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44dff6-0677-4766-b59e-841675a48680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:49.453592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44dff6-0677-4766-b59e-841675a48680 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:49.453863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44dff6-0677-4766-b59e-841675a48680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:49.454497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44dff6-0677-4766-b59e-841675a48680 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:49.454803 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44dff6-0677-4766-b59e-841675a48680 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:49.456432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af44dff6-0677-4766-b59e-841675a48680 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:49.468298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1214/2432] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:13:49.472250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5bb66-25e0-4f65-ab4d-eb8f6896a0ab admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:49.482595 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5bb66-25e0-4f65-ab4d-eb8f6896a0ab admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:49.488319 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbbe34a-529a-4f19-b1dc-b539db424bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:49.488919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbe34a-529a-4f19-b1dc-b539db424bdb admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:49.489192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbe34a-529a-4f19-b1dc-b539db424bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:49.489455 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbbe34a-529a-4f19-b1dc-b539db424bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:49.490219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbe34a-529a-4f19-b1dc-b539db424bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:49.492787 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbbe34a-529a-4f19-b1dc-b539db424bdb admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:49.505566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1219/2433] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:49.510073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44dff6-0677-4766-b59e-841675a48680 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:49.522841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44dff6-0677-4766-b59e-841675a48680 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:49.529529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068afed-80dc-483d-8ac7-390bbdb3e834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:49.530289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068afed-80dc-483d-8ac7-390bbdb3e834 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:49.530565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068afed-80dc-483d-8ac7-390bbdb3e834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:49.530835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068afed-80dc-483d-8ac7-390bbdb3e834 admin admin] PATH_INFO: `/v3/projects/baefb9ad9314402f82134351fc00b359/users/57b851f6dec04fbc95a5d2e4f5680f85/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:49.551227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068afed-80dc-483d-8ac7-390bbdb3e834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baefb9ad9314402f82134351fc00b359, user_id=57b851f6dec04fbc95a5d2e4f5680f85, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:49.554943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068afed-80dc-483d-8ac7-390bbdb3e834 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:49.575859 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1215/2434] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:13:49 2022] PUT /identity/v3/projects/baefb9ad9314402f82134351fc00b359/users/57b851f6dec04fbc95a5d2e4f5680f85/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:13:49.586601 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597c128-4a7b-47d2-87bd-a22675551090 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:49.586891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597c128-4a7b-47d2-87bd-a22675551090 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:49.587168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597c128-4a7b-47d2-87bd-a22675551090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:49.620507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-a597c128-4a7b-47d2-87bd-a22675551090 None None] MFA Rules not processed for user `57b851f6dec04fbc95a5d2e4f5680f85`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:49.621923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a597c128-4a7b-47d2-87bd-a22675551090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:49.690981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1220/2435] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:49.696095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068afed-80dc-483d-8ac7-390bbdb3e834 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:49.756867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068afed-80dc-483d-8ac7-390bbdb3e834 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:49.761754 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca4e00-aa6e-4654-a739-c5e5d4480071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:49.762359 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca4e00-aa6e-4654-a739-c5e5d4480071 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:49.762630 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca4e00-aa6e-4654-a739-c5e5d4480071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:49.762892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ca4e00-aa6e-4654-a739-c5e5d4480071 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:49.763566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca4e00-aa6e-4654-a739-c5e5d4480071 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:49.765476 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ca4e00-aa6e-4654-a739-c5e5d4480071 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:49.797448 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1216/2436] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:13:49 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:13:49.800192 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597c128-4a7b-47d2-87bd-a22675551090 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:49.813042 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597c128-4a7b-47d2-87bd-a22675551090 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:49.891772 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9d134-caa1-457b-8c94-4ef2ae70974a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:49.892489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9d134-caa1-457b-8c94-4ef2ae70974a admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:49.892793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9d134-caa1-457b-8c94-4ef2ae70974a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:49.893063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9d134-caa1-457b-8c94-4ef2ae70974a admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:49.893708 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9d134-caa1-457b-8c94-4ef2ae70974a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:49.895758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9d134-caa1-457b-8c94-4ef2ae70974a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:49.919182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1221/2437] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:13:49 2022] POST /identity/v3/users => generated 298 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:13:49.923484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca4e00-aa6e-4654-a739-c5e5d4480071 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:49.934808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ca4e00-aa6e-4654-a739-c5e5d4480071 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:49.940432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8443e-6e3f-471d-b255-d5d9f5b2c9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:49.941099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8443e-6e3f-471d-b255-d5d9f5b2c9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:49.941311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8443e-6e3f-471d-b255-d5d9f5b2c9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:49.941575 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8443e-6e3f-471d-b255-d5d9f5b2c9b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:49.942365 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8443e-6e3f-471d-b255-d5d9f5b2c9b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:49.944006 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e8443e-6e3f-471d-b255-d5d9f5b2c9b7 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:49.955871 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1217/2438] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:49.960223 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9d134-caa1-457b-8c94-4ef2ae70974a admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:49.970757 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9d134-caa1-457b-8c94-4ef2ae70974a admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:49.976006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82ba7e-138e-4bf0-9d37-4ad457cf9f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:49.976781 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82ba7e-138e-4bf0-9d37-4ad457cf9f31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:49.977116 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82ba7e-138e-4bf0-9d37-4ad457cf9f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:49.977446 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82ba7e-138e-4bf0-9d37-4ad457cf9f31 admin admin] PATH_INFO: `/v3/projects/5a6c9e37250d47269d36f537f8de6d62/users/0ad8f796dd5c4a449a7c67aecfdc607a/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:49.993241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82ba7e-138e-4bf0-9d37-4ad457cf9f31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6c9e37250d47269d36f537f8de6d62, user_id=0ad8f796dd5c4a449a7c67aecfdc607a, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:49.995994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82ba7e-138e-4bf0-9d37-4ad457cf9f31 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:50.009896 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1222/2439] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:13:49 2022] PUT /identity/v3/projects/5a6c9e37250d47269d36f537f8de6d62/users/0ad8f796dd5c4a449a7c67aecfdc607a/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:50.014194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8443e-6e3f-471d-b255-d5d9f5b2c9b7 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:50.025148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8443e-6e3f-471d-b255-d5d9f5b2c9b7 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:50.097844 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f9816-1b7a-4196-a30e-8fc39b5648d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:50.098622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f9816-1b7a-4196-a30e-8fc39b5648d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:50.099040 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f9816-1b7a-4196-a30e-8fc39b5648d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:50.099365 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f9816-1b7a-4196-a30e-8fc39b5648d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:50.100360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f9816-1b7a-4196-a30e-8fc39b5648d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:50.102876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5f9816-1b7a-4196-a30e-8fc39b5648d8 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:50.119462 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1218/2440] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:50.123958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82ba7e-138e-4bf0-9d37-4ad457cf9f31 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:50.152185 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82ba7e-138e-4bf0-9d37-4ad457cf9f31 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:50.175132 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccba2f7-6f1b-4ef0-8232-a9cfcfe1edb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:50.176778 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccba2f7-6f1b-4ef0-8232-a9cfcfe1edb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:50.177126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccba2f7-6f1b-4ef0-8232-a9cfcfe1edb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:50.177466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccba2f7-6f1b-4ef0-8232-a9cfcfe1edb5 admin admin] PATH_INFO: `/v3/projects/5a6c9e37250d47269d36f537f8de6d62/users/0ad8f796dd5c4a449a7c67aecfdc607a/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:50.184217 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccba2f7-6f1b-4ef0-8232-a9cfcfe1edb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a6c9e37250d47269d36f537f8de6d62, user_id=0ad8f796dd5c4a449a7c67aecfdc607a, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:50.190684 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccba2f7-6f1b-4ef0-8232-a9cfcfe1edb5 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:50.212082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1223/2441] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:13:50 2022] PUT /identity/v3/projects/5a6c9e37250d47269d36f537f8de6d62/users/0ad8f796dd5c4a449a7c67aecfdc607a/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:50.216276 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f757d-9979-4b2d-84ff-072f203d1496 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:50.216520 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f757d-9979-4b2d-84ff-072f203d1496 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:50.216799 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f757d-9979-4b2d-84ff-072f203d1496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:50.224701 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:50.251700 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-e99f757d-9979-4b2d-84ff-072f203d1496 None None] MFA Rules not processed for user `0ad8f796dd5c4a449a7c67aecfdc607a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:50.253327 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e99f757d-9979-4b2d-84ff-072f203d1496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:50.328566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1219/2442] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:50.339314 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccba2f7-6f1b-4ef0-8232-a9cfcfe1edb5 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:50.341141 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9ccba2f7-6f1b-4ef0-8232-a9cfcfe1edb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:50.368043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:50.402575 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccba2f7-6f1b-4ef0-8232-a9cfcfe1edb5 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:50.406302 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739cdb5-e484-4a86-840f-48512c1f130d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:50.406923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739cdb5-e484-4a86-840f-48512c1f130d service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:50.407202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739cdb5-e484-4a86-840f-48512c1f130d service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:50.407494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739cdb5-e484-4a86-840f-48512c1f130d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:50.417358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739cdb5-e484-4a86-840f-48512c1f130d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:50.418806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739cdb5-e484-4a86-840f-48512c1f130d service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:50.429225 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1224/2443] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:50.554356 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f757d-9979-4b2d-84ff-072f203d1496 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:50.565320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f757d-9979-4b2d-84ff-072f203d1496 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:50.569238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ef2cb-6b7b-4683-b32e-1b0aece0b663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:50.569899 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ef2cb-6b7b-4683-b32e-1b0aece0b663 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:50.570176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ef2cb-6b7b-4683-b32e-1b0aece0b663 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:50.570445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ef2cb-6b7b-4683-b32e-1b0aece0b663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:50.629681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ef2cb-6b7b-4683-b32e-1b0aece0b663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:50.631197 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ef2cb-6b7b-4683-b32e-1b0aece0b663 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:50.640444 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1220/2444] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:53.278470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739cdb5-e484-4a86-840f-48512c1f130d service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:53.356663 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739cdb5-e484-4a86-840f-48512c1f130d service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:53.369582 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c9e0a-e5c0-438c-822f-8d079cca2eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:53.370374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c9e0a-e5c0-438c-822f-8d079cca2eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:53.370652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c9e0a-e5c0-438c-822f-8d079cca2eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:53.370937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c9e0a-e5c0-438c-822f-8d079cca2eea admin admin] PATH_INFO: `/v3/users/57b851f6dec04fbc95a5d2e4f5680f85` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:53.372407 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c9e0a-e5c0-438c-822f-8d079cca2eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=57b851f6dec04fbc95a5d2e4f5680f85)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:53.374415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6c9e0a-e5c0-438c-822f-8d079cca2eea admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:53.434505 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5b6c9e0a-e5c0-438c-822f-8d079cca2eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57b851f6dec04fbc95a5d2e4f5680f85'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:53.444611 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5b6c9e0a-e5c0-438c-822f-8d079cca2eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57b851f6dec04fbc95a5d2e4f5680f85'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:53.445464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5b6c9e0a-e5c0-438c-822f-8d079cca2eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57b851f6dec04fbc95a5d2e4f5680f85'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:53.446964 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5b6c9e0a-e5c0-438c-822f-8d079cca2eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57b851f6dec04fbc95a5d2e4f5680f85'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:53.458627 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5b6c9e0a-e5c0-438c-822f-8d079cca2eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57b851f6dec04fbc95a5d2e4f5680f85'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:53.481278 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1225/2445] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:13:53 2022] DELETE /identity/v3/users/57b851f6dec04fbc95a5d2e4f5680f85 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:53.488546 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574ef36-7d83-46f3-bf1b-13774fb3a063 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:53.489041 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574ef36-7d83-46f3-bf1b-13774fb3a063 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:53.489145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f574ef36-7d83-46f3-bf1b-13774fb3a063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:53.509626 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f574ef36-7d83-46f3-bf1b-13774fb3a063 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:53.510848 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f574ef36-7d83-46f3-bf1b-13774fb3a063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:53.577909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1221/2446] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:53.587561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c9e0a-e5c0-438c-822f-8d079cca2eea admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:53.589197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5b6c9e0a-e5c0-438c-822f-8d079cca2eea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:53.646619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:53.660831 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c9e0a-e5c0-438c-822f-8d079cca2eea admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:53.664895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d1d2a-24f9-4695-b232-b4ee6f51db74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:53.665614 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d1d2a-24f9-4695-b232-b4ee6f51db74 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:53.666013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d1d2a-24f9-4695-b232-b4ee6f51db74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:53.666346 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d1d2a-24f9-4695-b232-b4ee6f51db74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:53.676128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d1d2a-24f9-4695-b232-b4ee6f51db74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:53.678011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d1d2a-24f9-4695-b232-b4ee6f51db74 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:53.689517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1226/2447] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:13:53.889765 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7af57-2bd0-48f1-86f2-64bcdbbe2ddc None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:53.890047 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7af57-2bd0-48f1-86f2-64bcdbbe2ddc None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:53.890325 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7af57-2bd0-48f1-86f2-64bcdbbe2ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:53.912294 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-edb7af57-2bd0-48f1-86f2-64bcdbbe2ddc None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:53.914194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-edb7af57-2bd0-48f1-86f2-64bcdbbe2ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:53.939070 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:53.990345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1222/2448] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Sep 8 16:13:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:54.042099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d1d2a-24f9-4695-b232-b4ee6f51db74 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:54.043571 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-149d1d2a-24f9-4695-b232-b4ee6f51db74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:54.071413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7af57-2bd0-48f1-86f2-64bcdbbe2ddc None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:54.073493 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-edb7af57-2bd0-48f1-86f2-64bcdbbe2ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:54.105058 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7af57-2bd0-48f1-86f2-64bcdbbe2ddc None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:54.109900 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405629c4-da89-4cd5-9265-6c2e15ee1736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:54.110536 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405629c4-da89-4cd5-9265-6c2e15ee1736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:54.110808 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405629c4-da89-4cd5-9265-6c2e15ee1736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:54.111075 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405629c4-da89-4cd5-9265-6c2e15ee1736 admin admin] PATH_INFO: `/v3/projects/baefb9ad9314402f82134351fc00b359` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:54.112235 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405629c4-da89-4cd5-9265-6c2e15ee1736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baefb9ad9314402f82134351fc00b359)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:54.114155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405629c4-da89-4cd5-9265-6c2e15ee1736 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:54.130963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d1d2a-24f9-4695-b232-b4ee6f51db74 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:54.136042 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5a58a-ca46-403e-b8a7-a3ced82daa7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:54.136650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5a58a-ca46-403e-b8a7-a3ced82daa7e service swift] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:54.136964 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5a58a-ca46-403e-b8a7-a3ced82daa7e service swift] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:54.137239 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5a58a-ca46-403e-b8a7-a3ced82daa7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:54.146532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb5a58a-ca46-403e-b8a7-a3ced82daa7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:54.148337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb5a58a-ca46-403e-b8a7-a3ced82daa7e service swift] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:54.157208 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1227/2449] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:13:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:13:54.162550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-405629c4-da89-4cd5-9265-6c2e15ee1736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baefb9ad9314402f82134351fc00b359', 'request_id': 'req-405629c4-da89-4cd5-9265-6c2e15ee1736'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:54.177582 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-405629c4-da89-4cd5-9265-6c2e15ee1736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baefb9ad9314402f82134351fc00b359', 'request_id': 'req-405629c4-da89-4cd5-9265-6c2e15ee1736'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:54.202831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-405629c4-da89-4cd5-9265-6c2e15ee1736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baefb9ad9314402f82134351fc00b359', 'request_id': 'req-405629c4-da89-4cd5-9265-6c2e15ee1736'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:54.225464 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-405629c4-da89-4cd5-9265-6c2e15ee1736 admin admin] The token cache is being invalidate because project baefb9ad9314402f82134351fc00b359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:13:54.226975 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-405629c4-da89-4cd5-9265-6c2e15ee1736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:54.231957 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-405629c4-da89-4cd5-9265-6c2e15ee1736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:54.233586 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1223/2450] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:13:54 2022] DELETE /identity/v3/projects/baefb9ad9314402f82134351fc00b359 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:54.237766 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5a58a-ca46-403e-b8a7-a3ced82daa7e service swift] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:54.251747 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5a58a-ca46-403e-b8a7-a3ced82daa7e service swift] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:54.324981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5da65-4ee4-4685-bf35-da7dad329879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:54.325699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5da65-4ee4-4685-bf35-da7dad329879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:54.325993 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5da65-4ee4-4685-bf35-da7dad329879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:54.326260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5da65-4ee4-4685-bf35-da7dad329879 admin admin] PATH_INFO: `/v3/users/0ad8f796dd5c4a449a7c67aecfdc607a` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:54.327413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d5da65-4ee4-4685-bf35-da7dad329879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ad8f796dd5c4a449a7c67aecfdc607a)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:54.330095 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d5da65-4ee4-4685-bf35-da7dad329879 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:54.389221 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b7d5da65-4ee4-4685-bf35-da7dad329879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ad8f796dd5c4a449a7c67aecfdc607a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:54.398144 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b7d5da65-4ee4-4685-bf35-da7dad329879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ad8f796dd5c4a449a7c67aecfdc607a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:54.399232 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b7d5da65-4ee4-4685-bf35-da7dad329879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ad8f796dd5c4a449a7c67aecfdc607a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:54.399999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b7d5da65-4ee4-4685-bf35-da7dad329879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ad8f796dd5c4a449a7c67aecfdc607a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:54.409445 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b7d5da65-4ee4-4685-bf35-da7dad329879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ad8f796dd5c4a449a7c67aecfdc607a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:54.425184 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:54.433428 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1228/2451] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:13:54 2022] DELETE /identity/v3/users/0ad8f796dd5c4a449a7c67aecfdc607a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:13:54.774915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405629c4-da89-4cd5-9265-6c2e15ee1736 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:54.776501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-405629c4-da89-4cd5-9265-6c2e15ee1736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:54.790957 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405629c4-da89-4cd5-9265-6c2e15ee1736 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:54.866387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc28c81-0c20-4b99-9041-7217d70810ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:54.867167 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc28c81-0c20-4b99-9041-7217d70810ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:54.867554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc28c81-0c20-4b99-9041-7217d70810ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:54.867902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc28c81-0c20-4b99-9041-7217d70810ca admin admin] PATH_INFO: `/v3/projects/5a6c9e37250d47269d36f537f8de6d62` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:54.869104 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc28c81-0c20-4b99-9041-7217d70810ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a6c9e37250d47269d36f537f8de6d62)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:54.871128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc28c81-0c20-4b99-9041-7217d70810ca admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:54.932814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0fc28c81-0c20-4b99-9041-7217d70810ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a6c9e37250d47269d36f537f8de6d62', 'request_id': 'req-0fc28c81-0c20-4b99-9041-7217d70810ca'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:54.942599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0fc28c81-0c20-4b99-9041-7217d70810ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a6c9e37250d47269d36f537f8de6d62', 'request_id': 'req-0fc28c81-0c20-4b99-9041-7217d70810ca'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:54.944288 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:54.956554 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0fc28c81-0c20-4b99-9041-7217d70810ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a6c9e37250d47269d36f537f8de6d62', 'request_id': 'req-0fc28c81-0c20-4b99-9041-7217d70810ca'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:54.971431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0fc28c81-0c20-4b99-9041-7217d70810ca admin admin] The token cache is being invalidate because project 5a6c9e37250d47269d36f537f8de6d62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:13:54.971826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0fc28c81-0c20-4b99-9041-7217d70810ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:54.972600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0fc28c81-0c20-4b99-9041-7217d70810ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:13:54.974144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1224/2452] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:13:54 2022] DELETE /identity/v3/projects/5a6c9e37250d47269d36f537f8de6d62 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:13:54.979999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfb2b5-ae49-4213-9d86-7e1909bab70c None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:54.980316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfb2b5-ae49-4213-9d86-7e1909bab70c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:54.980591 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddfb2b5-ae49-4213-9d86-7e1909bab70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:55.008233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-fddfb2b5-ae49-4213-9d86-7e1909bab70c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:55.009959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-fddfb2b5-ae49-4213-9d86-7e1909bab70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:55.083906 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1229/2453] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:55.090683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199a47f-ef71-4c0a-b199-90a62eab1b92 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:55.090949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199a47f-ef71-4c0a-b199-90a62eab1b92 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:55.091219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199a47f-ef71-4c0a-b199-90a62eab1b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:55.123760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-1199a47f-ef71-4c0a-b199-90a62eab1b92 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:55.124999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-1199a47f-ef71-4c0a-b199-90a62eab1b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:55.136506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1225/2454] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:55.137799 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfb2b5-ae49-4213-9d86-7e1909bab70c None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:55.150156 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddfb2b5-ae49-4213-9d86-7e1909bab70c None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:55.155227 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f6f46-17c9-475e-a9c9-717e05900e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:55.156095 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f6f46-17c9-475e-a9c9-717e05900e43 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:55.156370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f6f46-17c9-475e-a9c9-717e05900e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:55.156688 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f6f46-17c9-475e-a9c9-717e05900e43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:55.157566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f6f46-17c9-475e-a9c9-717e05900e43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:13:55.157869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f6f46-17c9-475e-a9c9-717e05900e43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:55.159546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f6f46-17c9-475e-a9c9-717e05900e43 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:55.178706 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1230/2455] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:13:55.182880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199a47f-ef71-4c0a-b199-90a62eab1b92 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:55.201542 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199a47f-ef71-4c0a-b199-90a62eab1b92 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:55.206392 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79080c-be1f-47c4-9aec-801aba05ac27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:55.207078 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79080c-be1f-47c4-9aec-801aba05ac27 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:55.207415 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79080c-be1f-47c4-9aec-801aba05ac27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:55.207776 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79080c-be1f-47c4-9aec-801aba05ac27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:55.208638 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79080c-be1f-47c4-9aec-801aba05ac27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:55.213324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79080c-be1f-47c4-9aec-801aba05ac27 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:55.258995 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f6f46-17c9-475e-a9c9-717e05900e43 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:55.260217 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1226/2456] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:13:55 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:13:55.279507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f6f46-17c9-475e-a9c9-717e05900e43 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:55.405666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbda33-1d9e-416d-baf0-723cdd206a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:55.407356 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbda33-1d9e-416d-baf0-723cdd206a0c admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:55.408005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbda33-1d9e-416d-baf0-723cdd206a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:55.408509 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbda33-1d9e-416d-baf0-723cdd206a0c admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:55.409249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbda33-1d9e-416d-baf0-723cdd206a0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:55.411692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babbda33-1d9e-416d-baf0-723cdd206a0c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:55.446893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1231/2457] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:13:55 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:13:55.450989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79080c-be1f-47c4-9aec-801aba05ac27 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:55.465300 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79080c-be1f-47c4-9aec-801aba05ac27 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:55.470335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf01663-7638-4596-8f44-41d7f33b38e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:55.471103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf01663-7638-4596-8f44-41d7f33b38e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:55.471470 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf01663-7638-4596-8f44-41d7f33b38e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:55.471834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf01663-7638-4596-8f44-41d7f33b38e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:55.472552 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf01663-7638-4596-8f44-41d7f33b38e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:55.474655 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf01663-7638-4596-8f44-41d7f33b38e6 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:55.487927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1227/2458] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:55.492702 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbda33-1d9e-416d-baf0-723cdd206a0c admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:55.504428 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbda33-1d9e-416d-baf0-723cdd206a0c admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:55.509969 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c0930-1759-4119-8cd8-48f091e8b1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:55.510688 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c0930-1759-4119-8cd8-48f091e8b1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:55.511031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c0930-1759-4119-8cd8-48f091e8b1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:55.511362 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c0930-1759-4119-8cd8-48f091e8b1e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:55.512279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c0930-1759-4119-8cd8-48f091e8b1e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:55.515571 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337c0930-1759-4119-8cd8-48f091e8b1e1 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:55.528723 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1232/2459] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:55.533476 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf01663-7638-4596-8f44-41d7f33b38e6 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:55.546927 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf01663-7638-4596-8f44-41d7f33b38e6 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:55.552202 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0ccae-6014-4530-a425-48b343573311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:55.553067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0ccae-6014-4530-a425-48b343573311 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:55.553411 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0ccae-6014-4530-a425-48b343573311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:55.553756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e0ccae-6014-4530-a425-48b343573311 admin admin] PATH_INFO: `/v3/projects/6b98ebea47e2463584d7a861471f78e8/users/31039d712a394d1d9741544be6c45f0d/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:55.584278 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e0ccae-6014-4530-a425-48b343573311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b98ebea47e2463584d7a861471f78e8, user_id=31039d712a394d1d9741544be6c45f0d, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:55.587581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e0ccae-6014-4530-a425-48b343573311 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:55.608282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1228/2460] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:13:55 2022] PUT /identity/v3/projects/6b98ebea47e2463584d7a861471f78e8/users/31039d712a394d1d9741544be6c45f0d/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:13:55.618790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03affa-d7a4-44b7-bc57-c666bdc38e42 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:55.619108 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03affa-d7a4-44b7-bc57-c666bdc38e42 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:55.619337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03affa-d7a4-44b7-bc57-c666bdc38e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:55.651548 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-fb03affa-d7a4-44b7-bc57-c666bdc38e42 None None] MFA Rules not processed for user `31039d712a394d1d9741544be6c45f0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:55.652774 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-fb03affa-d7a4-44b7-bc57-c666bdc38e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:55.727311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:55.755525 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1233/2461] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:55.759672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0ccae-6014-4530-a425-48b343573311 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:55.837920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e0ccae-6014-4530-a425-48b343573311 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:55.842914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5365-9cb2-4e6b-9b09-384a2006af44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:55.843641 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5365-9cb2-4e6b-9b09-384a2006af44 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:55.843986 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5365-9cb2-4e6b-9b09-384a2006af44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:55.844342 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b5365-9cb2-4e6b-9b09-384a2006af44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:55.847503 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b5365-9cb2-4e6b-9b09-384a2006af44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:55.849844 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b5365-9cb2-4e6b-9b09-384a2006af44 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:55.900770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1229/2462] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:13:55 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:13:55.901960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03affa-d7a4-44b7-bc57-c666bdc38e42 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:55.917070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03affa-d7a4-44b7-bc57-c666bdc38e42 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:55.990604 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be5ba1-02c7-41c6-b22c-d873e84ce0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:55.991548 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be5ba1-02c7-41c6-b22c-d873e84ce0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:55.991795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be5ba1-02c7-41c6-b22c-d873e84ce0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:55.992051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be5ba1-02c7-41c6-b22c-d873e84ce0e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:55.992728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be5ba1-02c7-41c6-b22c-d873e84ce0e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:55.994823 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be5ba1-02c7-41c6-b22c-d873e84ce0e4 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:56.023515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1234/2463] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:13:55 2022] POST /identity/v3/users => generated 298 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:13:56.025658 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5365-9cb2-4e6b-9b09-384a2006af44 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:56.036266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b5365-9cb2-4e6b-9b09-384a2006af44 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:56.041523 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae29bb-e6c8-4fe3-8190-5fe0125e882f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:56.042241 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae29bb-e6c8-4fe3-8190-5fe0125e882f admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:56.042594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae29bb-e6c8-4fe3-8190-5fe0125e882f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:56.042931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae29bb-e6c8-4fe3-8190-5fe0125e882f admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:56.043617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae29bb-e6c8-4fe3-8190-5fe0125e882f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:56.045233 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae29bb-e6c8-4fe3-8190-5fe0125e882f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:56.057449 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1230/2464] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:56.061726 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be5ba1-02c7-41c6-b22c-d873e84ce0e4 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:56.072728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be5ba1-02c7-41c6-b22c-d873e84ce0e4 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:56.077843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e25f3-8b6b-419a-89a6-b792b0651285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:56.078514 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e25f3-8b6b-419a-89a6-b792b0651285 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:56.078791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e25f3-8b6b-419a-89a6-b792b0651285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:56.079061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e25f3-8b6b-419a-89a6-b792b0651285 admin admin] PATH_INFO: `/v3/projects/1102c506f3b94f9b9b048e7bc42d4279/users/ecd46f564df6410292d9d4f7641c844c/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:56.097299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e25f3-8b6b-419a-89a6-b792b0651285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1102c506f3b94f9b9b048e7bc42d4279, user_id=ecd46f564df6410292d9d4f7641c844c, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:56.100714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e25f3-8b6b-419a-89a6-b792b0651285 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:56.118895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1235/2465] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:13:56 2022] PUT /identity/v3/projects/1102c506f3b94f9b9b048e7bc42d4279/users/ecd46f564df6410292d9d4f7641c844c/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:13:56.122872 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae29bb-e6c8-4fe3-8190-5fe0125e882f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:56.137410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae29bb-e6c8-4fe3-8190-5fe0125e882f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:56.203066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:56.211755 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b9365-6ed2-4a09-b157-d471ace27d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:56.212473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b9365-6ed2-4a09-b157-d471ace27d69 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:56.215887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b9365-6ed2-4a09-b157-d471ace27d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:56.216249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b9365-6ed2-4a09-b157-d471ace27d69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:56.217185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b9365-6ed2-4a09-b157-d471ace27d69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:56.219251 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b9365-6ed2-4a09-b157-d471ace27d69 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:56.229812 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1231/2466] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:13:56.234064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e25f3-8b6b-419a-89a6-b792b0651285 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:56.245219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e25f3-8b6b-419a-89a6-b792b0651285 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:56.250120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9a8b5-0036-4c95-a68f-572c7137b8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:56.250811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9a8b5-0036-4c95-a68f-572c7137b8f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:56.251081 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9a8b5-0036-4c95-a68f-572c7137b8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:56.251346 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e9a8b5-0036-4c95-a68f-572c7137b8f5 admin admin] PATH_INFO: `/v3/projects/1102c506f3b94f9b9b048e7bc42d4279/users/ecd46f564df6410292d9d4f7641c844c/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:56.253477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9a8b5-0036-4c95-a68f-572c7137b8f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1102c506f3b94f9b9b048e7bc42d4279, user_id=ecd46f564df6410292d9d4f7641c844c, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:56.257200 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e9a8b5-0036-4c95-a68f-572c7137b8f5 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:56.268864 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1236/2467] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:13:56 2022] PUT /identity/v3/projects/1102c506f3b94f9b9b048e7bc42d4279/users/ecd46f564df6410292d9d4f7641c844c/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:13:56.340016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2f477-3250-4090-999c-7ffbd79591cf None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:56.340341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2f477-3250-4090-999c-7ffbd79591cf None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:56.340558 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2f477-3250-4090-999c-7ffbd79591cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:56.369627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-7aa2f477-3250-4090-999c-7ffbd79591cf None None] MFA Rules not processed for user `ecd46f564df6410292d9d4f7641c844c`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:13:56.370759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-7aa2f477-3250-4090-999c-7ffbd79591cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:56.466606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1232/2468] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:13:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:13:56.474763 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9a8b5-0036-4c95-a68f-572c7137b8f5 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:13:56.476840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f8e9a8b5-0036-4c95-a68f-572c7137b8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:13:56.564565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e9a8b5-0036-4c95-a68f-572c7137b8f5 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:13:56.569284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e6d88-a58b-45cc-b3c7-e3a5151ea287 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:13:56.570003 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e6d88-a58b-45cc-b3c7-e3a5151ea287 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:13:56.570284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e6d88-a58b-45cc-b3c7-e3a5151ea287 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:13:56.570556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689e6d88-a58b-45cc-b3c7-e3a5151ea287 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:13:56.642112 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e6d88-a58b-45cc-b3c7-e3a5151ea287 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:13:56.643478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689e6d88-a58b-45cc-b3c7-e3a5151ea287 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:13:56.647217 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:13:56.656824 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1237/2469] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:13:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:14:01.353423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2f477-3250-4090-999c-7ffbd79591cf None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:01.354942 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-7aa2f477-3250-4090-999c-7ffbd79591cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:01.374698 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2f477-3250-4090-999c-7ffbd79591cf None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:01.379094 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f6f3c-8513-438e-b446-0a8bbff67057 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:01.379759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f6f3c-8513-438e-b446-0a8bbff67057 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:01.380037 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f6f3c-8513-438e-b446-0a8bbff67057 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:01.380329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f6f3c-8513-438e-b446-0a8bbff67057 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:01.394991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f6f3c-8513-438e-b446-0a8bbff67057 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:01.396624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f6f3c-8513-438e-b446-0a8bbff67057 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:01.412021 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1233/2470] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:14:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:14:06.568541 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e6d88-a58b-45cc-b3c7-e3a5151ea287 service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:06.635559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689e6d88-a58b-45cc-b3c7-e3a5151ea287 service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:06.640254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f2e20-78e8-4834-84d0-147a5837b17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:06.640890 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f2e20-78e8-4834-84d0-147a5837b17e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:06.641166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f2e20-78e8-4834-84d0-147a5837b17e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:06.641440 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f2e20-78e8-4834-84d0-147a5837b17e admin admin] PATH_INFO: `/v3/users/31039d712a394d1d9741544be6c45f0d` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:06.642578 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f2e20-78e8-4834-84d0-147a5837b17e admin admin] RBAC: Authorizing `identity:delete_user(user_id=31039d712a394d1d9741544be6c45f0d)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:06.644561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403f2e20-78e8-4834-84d0-147a5837b17e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:06.694097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-403f2e20-78e8-4834-84d0-147a5837b17e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31039d712a394d1d9741544be6c45f0d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:06.702443 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-403f2e20-78e8-4834-84d0-147a5837b17e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31039d712a394d1d9741544be6c45f0d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:06.703146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-403f2e20-78e8-4834-84d0-147a5837b17e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31039d712a394d1d9741544be6c45f0d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:06.703809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-403f2e20-78e8-4834-84d0-147a5837b17e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31039d712a394d1d9741544be6c45f0d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:06.712790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-403f2e20-78e8-4834-84d0-147a5837b17e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31039d712a394d1d9741544be6c45f0d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:06.740765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1238/2471] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:14:06 2022] DELETE /identity/v3/users/31039d712a394d1d9741544be6c45f0d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:06.745077 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040576c-b5e6-4012-b382-9eeedd9e27ac None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:06.745581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040576c-b5e6-4012-b382-9eeedd9e27ac None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:06.745889 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040576c-b5e6-4012-b382-9eeedd9e27ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:06.777597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-5040576c-b5e6-4012-b382-9eeedd9e27ac None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:06.779949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5040576c-b5e6-4012-b382-9eeedd9e27ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:06.845128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1234/2472] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:06.855125 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f2e20-78e8-4834-84d0-147a5837b17e admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:06.856759 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-403f2e20-78e8-4834-84d0-147a5837b17e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:06.929186 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f2e20-78e8-4834-84d0-147a5837b17e admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:06.932976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc4320-f435-4c96-8bf8-e7e302a3aef8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:06.933597 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc4320-f435-4c96-8bf8-e7e302a3aef8 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:06.933871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc4320-f435-4c96-8bf8-e7e302a3aef8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:06.934137 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc4320-f435-4c96-8bf8-e7e302a3aef8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:06.953517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc4320-f435-4c96-8bf8-e7e302a3aef8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:06.954995 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc4320-f435-4c96-8bf8-e7e302a3aef8 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:06.971490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1239/2473] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:14:07.316502 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040576c-b5e6-4012-b382-9eeedd9e27ac None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:07.318508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-5040576c-b5e6-4012-b382-9eeedd9e27ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:07.346241 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040576c-b5e6-4012-b382-9eeedd9e27ac None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:07.353498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fdb45-2831-4df9-a484-c08b79f482dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:07.354274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fdb45-2831-4df9-a484-c08b79f482dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:07.354622 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fdb45-2831-4df9-a484-c08b79f482dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:07.355027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fdb45-2831-4df9-a484-c08b79f482dc admin admin] PATH_INFO: `/v3/projects/6b98ebea47e2463584d7a861471f78e8` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:07.357082 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53fdb45-2831-4df9-a484-c08b79f482dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b98ebea47e2463584d7a861471f78e8)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:07.359524 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53fdb45-2831-4df9-a484-c08b79f482dc admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:07.453539 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f53fdb45-2831-4df9-a484-c08b79f482dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b98ebea47e2463584d7a861471f78e8', 'request_id': 'req-f53fdb45-2831-4df9-a484-c08b79f482dc'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:07.463619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f53fdb45-2831-4df9-a484-c08b79f482dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b98ebea47e2463584d7a861471f78e8', 'request_id': 'req-f53fdb45-2831-4df9-a484-c08b79f482dc'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:07.467207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:07.473864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f53fdb45-2831-4df9-a484-c08b79f482dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b98ebea47e2463584d7a861471f78e8', 'request_id': 'req-f53fdb45-2831-4df9-a484-c08b79f482dc'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:07.499913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f53fdb45-2831-4df9-a484-c08b79f482dc admin admin] The token cache is being invalidate because project 6b98ebea47e2463584d7a861471f78e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:14:07.500231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f53fdb45-2831-4df9-a484-c08b79f482dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:07.503925 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f53fdb45-2831-4df9-a484-c08b79f482dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:07.509212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1235/2474] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:14:07 2022] DELETE /identity/v3/projects/6b98ebea47e2463584d7a861471f78e8 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:07.512390 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc4320-f435-4c96-8bf8-e7e302a3aef8 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:07.576829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc4320-f435-4c96-8bf8-e7e302a3aef8 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:07.627977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:07.698260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60d6010-1067-473f-96bf-881fe2a4036c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:07.699034 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d6010-1067-473f-96bf-881fe2a4036c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:07.699387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d6010-1067-473f-96bf-881fe2a4036c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:07.699775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60d6010-1067-473f-96bf-881fe2a4036c admin admin] PATH_INFO: `/v3/users/ecd46f564df6410292d9d4f7641c844c` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:07.701055 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d6010-1067-473f-96bf-881fe2a4036c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecd46f564df6410292d9d4f7641c844c)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:07.704008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60d6010-1067-473f-96bf-881fe2a4036c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:07.761089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e60d6010-1067-473f-96bf-881fe2a4036c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecd46f564df6410292d9d4f7641c844c'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:07.773299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e60d6010-1067-473f-96bf-881fe2a4036c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecd46f564df6410292d9d4f7641c844c'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:07.774084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e60d6010-1067-473f-96bf-881fe2a4036c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecd46f564df6410292d9d4f7641c844c'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:07.774700 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e60d6010-1067-473f-96bf-881fe2a4036c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecd46f564df6410292d9d4f7641c844c'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:07.783607 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e60d6010-1067-473f-96bf-881fe2a4036c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecd46f564df6410292d9d4f7641c844c'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:07.805595 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1240/2475] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:14:07 2022] DELETE /identity/v3/users/ecd46f564df6410292d9d4f7641c844c => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:08.107015 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fdb45-2831-4df9-a484-c08b79f482dc admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:08.108723 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f53fdb45-2831-4df9-a484-c08b79f482dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:08.128289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fdb45-2831-4df9-a484-c08b79f482dc admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:08.201790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe7b2e-4cec-4abe-8a4a-6f866a56473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:08.202357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe7b2e-4cec-4abe-8a4a-6f866a56473d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:08.202631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe7b2e-4cec-4abe-8a4a-6f866a56473d admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:08.202899 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe7b2e-4cec-4abe-8a4a-6f866a56473d admin admin] PATH_INFO: `/v3/projects/1102c506f3b94f9b9b048e7bc42d4279` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:08.204300 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe7b2e-4cec-4abe-8a4a-6f866a56473d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1102c506f3b94f9b9b048e7bc42d4279)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:08.206360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe7b2e-4cec-4abe-8a4a-6f866a56473d admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:08.260431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-43fe7b2e-4cec-4abe-8a4a-6f866a56473d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1102c506f3b94f9b9b048e7bc42d4279', 'request_id': 'req-43fe7b2e-4cec-4abe-8a4a-6f866a56473d'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:08.269085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-43fe7b2e-4cec-4abe-8a4a-6f866a56473d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1102c506f3b94f9b9b048e7bc42d4279', 'request_id': 'req-43fe7b2e-4cec-4abe-8a4a-6f866a56473d'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:08.275680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-43fe7b2e-4cec-4abe-8a4a-6f866a56473d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1102c506f3b94f9b9b048e7bc42d4279', 'request_id': 'req-43fe7b2e-4cec-4abe-8a4a-6f866a56473d'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:08.289131 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-43fe7b2e-4cec-4abe-8a4a-6f866a56473d admin admin] The token cache is being invalidate because project 1102c506f3b94f9b9b048e7bc42d4279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:14:08.289421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-43fe7b2e-4cec-4abe-8a4a-6f866a56473d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:08.290243 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-43fe7b2e-4cec-4abe-8a4a-6f866a56473d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:08.291867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1236/2476] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:14:08 2022] DELETE /identity/v3/projects/1102c506f3b94f9b9b048e7bc42d4279 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:08.299068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd92a0-e1c6-4495-9f5f-df46a69ddbcc None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:08.299626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd92a0-e1c6-4495-9f5f-df46a69ddbcc None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:08.300064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd92a0-e1c6-4495-9f5f-df46a69ddbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:08.323028 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-acbd92a0-e1c6-4495-9f5f-df46a69ddbcc None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:08.324515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-acbd92a0-e1c6-4495-9f5f-df46a69ddbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:08.389438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1241/2477] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:08.398077 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3e776-8284-418d-b3ab-429f490d2327 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:08.398610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3e776-8284-418d-b3ab-429f490d2327 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:08.399069 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3e776-8284-418d-b3ab-429f490d2327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:08.419033 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f9f3e776-8284-418d-b3ab-429f490d2327 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:08.420264 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f9f3e776-8284-418d-b3ab-429f490d2327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:08.428234 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1237/2478] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:08.432680 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd92a0-e1c6-4495-9f5f-df46a69ddbcc None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:08.443040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd92a0-e1c6-4495-9f5f-df46a69ddbcc None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:08.448088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48a0a1-a613-4c5c-9151-0242cc60a7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:08.448703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48a0a1-a613-4c5c-9151-0242cc60a7fc admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:08.448999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48a0a1-a613-4c5c-9151-0242cc60a7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:08.449247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48a0a1-a613-4c5c-9151-0242cc60a7fc admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:08.450012 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48a0a1-a613-4c5c-9151-0242cc60a7fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:14:08.450316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48a0a1-a613-4c5c-9151-0242cc60a7fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:08.452070 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48a0a1-a613-4c5c-9151-0242cc60a7fc admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:08.464344 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1242/2479] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:14:08.468883 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3e776-8284-418d-b3ab-429f490d2327 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:08.478723 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3e776-8284-418d-b3ab-429f490d2327 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:08.483759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c65d9-af5c-4ea2-bde3-f9199ff7b49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:08.484357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c65d9-af5c-4ea2-bde3-f9199ff7b49f admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:08.484628 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c65d9-af5c-4ea2-bde3-f9199ff7b49f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:08.484901 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c65d9-af5c-4ea2-bde3-f9199ff7b49f admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:08.485535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c65d9-af5c-4ea2-bde3-f9199ff7b49f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:08.487418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c65d9-af5c-4ea2-bde3-f9199ff7b49f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:08.520979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1238/2480] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:14:08 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:14:08.522227 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48a0a1-a613-4c5c-9151-0242cc60a7fc admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:08.532592 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48a0a1-a613-4c5c-9151-0242cc60a7fc admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:08.559477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:08.593943 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b584eb-4cd1-4645-a873-2486139a849a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:08.594637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b584eb-4cd1-4645-a873-2486139a849a admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:08.594842 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b584eb-4cd1-4645-a873-2486139a849a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:08.595114 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b584eb-4cd1-4645-a873-2486139a849a admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:08.595791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b584eb-4cd1-4645-a873-2486139a849a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:08.597724 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b584eb-4cd1-4645-a873-2486139a849a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:08.629436 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1243/2481] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:14:08 2022] POST /identity/v3/users => generated 299 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:14:08.633987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c65d9-af5c-4ea2-bde3-f9199ff7b49f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:08.645071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c65d9-af5c-4ea2-bde3-f9199ff7b49f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:08.652796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1de58c-546f-49b7-907c-ab88a1f5d25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:08.653421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1de58c-546f-49b7-907c-ab88a1f5d25e admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:08.653718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1de58c-546f-49b7-907c-ab88a1f5d25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:08.653994 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1de58c-546f-49b7-907c-ab88a1f5d25e admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:08.654694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1de58c-546f-49b7-907c-ab88a1f5d25e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:08.656459 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1de58c-546f-49b7-907c-ab88a1f5d25e admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:08.671350 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1239/2482] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:14:08.673340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b584eb-4cd1-4645-a873-2486139a849a admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:08.684237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b584eb-4cd1-4645-a873-2486139a849a admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:08.689764 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a80a75-baab-4d28-8203-d7801dc396e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:08.690499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a80a75-baab-4d28-8203-d7801dc396e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:08.690842 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a80a75-baab-4d28-8203-d7801dc396e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:08.691181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a80a75-baab-4d28-8203-d7801dc396e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:08.692100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a80a75-baab-4d28-8203-d7801dc396e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:08.693851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a80a75-baab-4d28-8203-d7801dc396e8 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:08.706092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1244/2483] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:14:08.710816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1de58c-546f-49b7-907c-ab88a1f5d25e admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:08.720752 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1de58c-546f-49b7-907c-ab88a1f5d25e admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:08.725857 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b31366-8c14-4ce3-9b84-3413c93aeb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:08.726615 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b31366-8c14-4ce3-9b84-3413c93aeb5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:08.726888 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b31366-8c14-4ce3-9b84-3413c93aeb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:08.727158 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b31366-8c14-4ce3-9b84-3413c93aeb5d admin admin] PATH_INFO: `/v3/projects/5383f90aa743418aa5c96f02217c6b1a/users/869cf40ef8fa4cb4876de50f21a4ec83/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:08.743422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b31366-8c14-4ce3-9b84-3413c93aeb5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5383f90aa743418aa5c96f02217c6b1a, user_id=869cf40ef8fa4cb4876de50f21a4ec83, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:08.746163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b31366-8c14-4ce3-9b84-3413c93aeb5d admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:08.761322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1240/2484] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:14:08 2022] PUT /identity/v3/projects/5383f90aa743418aa5c96f02217c6b1a/users/869cf40ef8fa4cb4876de50f21a4ec83/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:14:08.772533 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cea367-2cee-45b1-a833-a4186740be54 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:08.772779 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cea367-2cee-45b1-a833-a4186740be54 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:08.773067 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cea367-2cee-45b1-a833-a4186740be54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:08.801145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-f2cea367-2cee-45b1-a833-a4186740be54 None None] MFA Rules not processed for user `869cf40ef8fa4cb4876de50f21a4ec83`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:08.802545 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f2cea367-2cee-45b1-a833-a4186740be54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:08.871137 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1245/2485] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:14:08 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:08.883320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b31366-8c14-4ce3-9b84-3413c93aeb5d admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:08.885227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c5b31366-8c14-4ce3-9b84-3413c93aeb5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:08.900942 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:08.953158 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b31366-8c14-4ce3-9b84-3413c93aeb5d admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:08.957289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def19af-1fde-4936-9bfc-62780d38ccc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:08.957964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def19af-1fde-4936-9bfc-62780d38ccc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:08.958259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def19af-1fde-4936-9bfc-62780d38ccc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:08.958532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def19af-1fde-4936-9bfc-62780d38ccc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:08.968730 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def19af-1fde-4936-9bfc-62780d38ccc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:08.971164 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def19af-1fde-4936-9bfc-62780d38ccc5 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:08.982699 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1241/2486] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:14:08 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:14:10.871003 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7322177-4a1c-4f3d-82a2-e744021448f7 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:10.871545 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7322177-4a1c-4f3d-82a2-e744021448f7 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:10.871545 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7322177-4a1c-4f3d-82a2-e744021448f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:10.891495 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-c7322177-4a1c-4f3d-82a2-e744021448f7 None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:10.892727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c7322177-4a1c-4f3d-82a2-e744021448f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:10.957448 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1246/2487] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Sep 8 16:14:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:10.972603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def19af-1fde-4936-9bfc-62780d38ccc5 service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:10.974236 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0def19af-1fde-4936-9bfc-62780d38ccc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:11.050367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def19af-1fde-4936-9bfc-62780d38ccc5 service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:11.057133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c9548-b6cb-4071-996a-4ea02fd36c40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:11.057768 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c9548-b6cb-4071-996a-4ea02fd36c40 service swift] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:11.058043 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c9548-b6cb-4071-996a-4ea02fd36c40 service swift] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:11.058307 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c9548-b6cb-4071-996a-4ea02fd36c40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:11.067866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c9548-b6cb-4071-996a-4ea02fd36c40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:11.069486 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c9548-b6cb-4071-996a-4ea02fd36c40 service swift] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:11.077906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1242/2488] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:14:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:14:17.179899 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7322177-4a1c-4f3d-82a2-e744021448f7 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:17.228935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:17.256101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7322177-4a1c-4f3d-82a2-e744021448f7 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:17.268111 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2004f-6604-46de-a52d-685bb1961d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:17.268920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2004f-6604-46de-a52d-685bb1961d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:17.269262 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2004f-6604-46de-a52d-685bb1961d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:17.269540 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2004f-6604-46de-a52d-685bb1961d35 admin admin] PATH_INFO: `/v3/users/869cf40ef8fa4cb4876de50f21a4ec83` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:17.270840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2004f-6604-46de-a52d-685bb1961d35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=869cf40ef8fa4cb4876de50f21a4ec83)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:17.272875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2004f-6604-46de-a52d-685bb1961d35 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:17.333453 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cac2004f-6604-46de-a52d-685bb1961d35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '869cf40ef8fa4cb4876de50f21a4ec83'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:17.341842 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cac2004f-6604-46de-a52d-685bb1961d35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '869cf40ef8fa4cb4876de50f21a4ec83'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:17.342588 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cac2004f-6604-46de-a52d-685bb1961d35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '869cf40ef8fa4cb4876de50f21a4ec83'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:17.343302 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cac2004f-6604-46de-a52d-685bb1961d35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '869cf40ef8fa4cb4876de50f21a4ec83'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:17.361500 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-cac2004f-6604-46de-a52d-685bb1961d35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '869cf40ef8fa4cb4876de50f21a4ec83'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:17.382861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1247/2489] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:14:17 2022] DELETE /identity/v3/users/869cf40ef8fa4cb4876de50f21a4ec83 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:17.387941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e975cb-692d-488e-a2d9-07ea5ec60692 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:17.388519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e975cb-692d-488e-a2d9-07ea5ec60692 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:17.388699 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e975cb-692d-488e-a2d9-07ea5ec60692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:17.413819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-d2e975cb-692d-488e-a2d9-07ea5ec60692 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:17.415428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d2e975cb-692d-488e-a2d9-07ea5ec60692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:17.479078 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1243/2490] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:17.488994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2004f-6604-46de-a52d-685bb1961d35 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:17.490560 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-cac2004f-6604-46de-a52d-685bb1961d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:17.565991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2004f-6604-46de-a52d-685bb1961d35 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:17.569712 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0827b8-88ab-45fc-86e0-ceeb7f46c2ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:17.570365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0827b8-88ab-45fc-86e0-ceeb7f46c2ae service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:17.570641 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0827b8-88ab-45fc-86e0-ceeb7f46c2ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:17.570913 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0827b8-88ab-45fc-86e0-ceeb7f46c2ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:17.581704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0827b8-88ab-45fc-86e0-ceeb7f46c2ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:17.583183 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0827b8-88ab-45fc-86e0-ceeb7f46c2ae service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:17.594516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1248/2491] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:14:17.879132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e975cb-692d-488e-a2d9-07ea5ec60692 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:17.880986 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d2e975cb-692d-488e-a2d9-07ea5ec60692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:17.899260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e975cb-692d-488e-a2d9-07ea5ec60692 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:17.908336 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:17.909388 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:17.910003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:17.910321 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1 admin admin] PATH_INFO: `/v3/projects/5383f90aa743418aa5c96f02217c6b1a` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:17.911896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5383f90aa743418aa5c96f02217c6b1a)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:17.914054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:17.963318 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:17.988067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5383f90aa743418aa5c96f02217c6b1a', 'request_id': 'req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:18.004650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5383f90aa743418aa5c96f02217c6b1a', 'request_id': 'req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:18.011423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5383f90aa743418aa5c96f02217c6b1a', 'request_id': 'req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:18.023374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1 admin admin] The token cache is being invalidate because project 5383f90aa743418aa5c96f02217c6b1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:14:18.023687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:18.024340 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:18.025852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1244/2492] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:14:17 2022] DELETE /identity/v3/projects/5383f90aa743418aa5c96f02217c6b1a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:27.296065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0dd15-86ee-45c1-8359-efbbd8475055 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:27.296573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0dd15-86ee-45c1-8359-efbbd8475055 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:27.296573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0dd15-86ee-45c1-8359-efbbd8475055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:27.319425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-d0c0dd15-86ee-45c1-8359-efbbd8475055 None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:27.320447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d0c0dd15-86ee-45c1-8359-efbbd8475055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:27.380113 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1249/2493] 10.222.0.27 () {58 vars in 967 bytes} [Thu Sep 8 16:14:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:27.392557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:27.393968 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:27.467854 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea53bc4-036a-4f2f-8e9e-7cbca87825a1 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:27.472934 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac03cf9-a869-4b19-a8a7-8f0479f2eb85 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:27.473541 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac03cf9-a869-4b19-a8a7-8f0479f2eb85 service swift] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:27.473816 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac03cf9-a869-4b19-a8a7-8f0479f2eb85 service swift] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:27.474086 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac03cf9-a869-4b19-a8a7-8f0479f2eb85 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:27.482162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac03cf9-a869-4b19-a8a7-8f0479f2eb85 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:27.483849 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac03cf9-a869-4b19-a8a7-8f0479f2eb85 service swift] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:27.491863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1245/2494] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:14:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:14:32.602765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0dd15-86ee-45c1-8359-efbbd8475055 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:32.604550 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d0c0dd15-86ee-45c1-8359-efbbd8475055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:32.614948 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:32.684893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0dd15-86ee-45c1-8359-efbbd8475055 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:32.691242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad1213-b468-4e8a-b67a-cfeff9311354 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:32.691878 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad1213-b468-4e8a-b67a-cfeff9311354 service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:32.692150 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad1213-b468-4e8a-b67a-cfeff9311354 service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:32.692411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad1213-b468-4e8a-b67a-cfeff9311354 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:32.694227 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-bbad1213-b468-4e8a-b67a-cfeff9311354 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:32.755200 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad1213-b468-4e8a-b67a-cfeff9311354 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:32.756749 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad1213-b468-4e8a-b67a-cfeff9311354 service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:32.766531 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1250/2495] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:14:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:14:37.489085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=41,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:39.094380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac03cf9-a869-4b19-a8a7-8f0479f2eb85 service swift] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:39.096042 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2ac03cf9-a869-4b19-a8a7-8f0479f2eb85 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:39.160093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac03cf9-a869-4b19-a8a7-8f0479f2eb85 service swift] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:39.164363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444270c-96cc-41e6-a666-b43036390f1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:39.164971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444270c-96cc-41e6-a666-b43036390f1f service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:39.165217 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444270c-96cc-41e6-a666-b43036390f1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:39.165959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444270c-96cc-41e6-a666-b43036390f1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:39.168011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0444270c-96cc-41e6-a666-b43036390f1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:39.234077 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444270c-96cc-41e6-a666-b43036390f1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:39.235569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444270c-96cc-41e6-a666-b43036390f1f service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:39.245417 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1246/2496] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:14:42.612955 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c93e2d-dee3-4bd6-9e52-c4945faa3a08 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:42.613780 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c93e2d-dee3-4bd6-9e52-c4945faa3a08 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:42.614059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c93e2d-dee3-4bd6-9e52-c4945faa3a08 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:42.615560 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1251/2497] 10.222.0.27 () {54 vars in 921 bytes} [Thu Sep 8 16:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:14:42.650024 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac640e-6a0d-4ee7-b22a-17852e896798 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:42.650510 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac640e-6a0d-4ee7-b22a-17852e896798 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:42.651125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac640e-6a0d-4ee7-b22a-17852e896798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:42.673621 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-88ac640e-6a0d-4ee7-b22a-17852e896798 None None] MFA Rules not processed for user `dd28bb31b55e45668212c9ab69ed3ac3`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:42.675011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-88ac640e-6a0d-4ee7-b22a-17852e896798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:42.696386 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c93e2d-dee3-4bd6-9e52-c4945faa3a08 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:42.700693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d6c93e2d-dee3-4bd6-9e52-c4945faa3a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:42.731837 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c93e2d-dee3-4bd6-9e52-c4945faa3a08 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:42.738219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d3ee3-e504-401e-935c-a6013f2c325a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:42.739240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d3ee3-e504-401e-935c-a6013f2c325a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:42.739569 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d3ee3-e504-401e-935c-a6013f2c325a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:42.740707 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d3ee3-e504-401e-935c-a6013f2c325a admin admin] PATH_INFO: `/v3/users/00b673d5c65d40d9a90cc3b579e83e7f` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:42.742390 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d3ee3-e504-401e-935c-a6013f2c325a admin admin] RBAC: Authorizing `identity:delete_user(user_id=00b673d5c65d40d9a90cc3b579e83e7f)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:42.744414 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4d3ee3-e504-401e-935c-a6013f2c325a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:42.773709 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1247/2498] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Sep 8 16:14:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:42.786811 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac640e-6a0d-4ee7-b22a-17852e896798 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:42.789043 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-88ac640e-6a0d-4ee7-b22a-17852e896798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:42.802668 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-9f4d3ee3-e504-401e-935c-a6013f2c325a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00b673d5c65d40d9a90cc3b579e83e7f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:42.803186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac640e-6a0d-4ee7-b22a-17852e896798 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:42.809746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b454b-0033-46c1-810d-e645e34c3b32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd28bb31b55e45668212c9ab69ed3ac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:42.810360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b454b-0033-46c1-810d-e645e34c3b32 service placement] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:42.811169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b454b-0033-46c1-810d-e645e34c3b32 service placement] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:42.811428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9b454b-0033-46c1-810d-e645e34c3b32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:42.811717 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-9f4d3ee3-e504-401e-935c-a6013f2c325a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00b673d5c65d40d9a90cc3b579e83e7f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:42.812678 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-9f4d3ee3-e504-401e-935c-a6013f2c325a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00b673d5c65d40d9a90cc3b579e83e7f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:42.813343 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-9f4d3ee3-e504-401e-935c-a6013f2c325a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00b673d5c65d40d9a90cc3b579e83e7f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:42.821111 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b454b-0033-46c1-810d-e645e34c3b32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:42.822723 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9b454b-0033-46c1-810d-e645e34c3b32 service placement] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:42.823710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-9f4d3ee3-e504-401e-935c-a6013f2c325a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00b673d5c65d40d9a90cc3b579e83e7f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:42.834030 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1248/2499] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:14:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:14:42.843313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1252/2500] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:14:42 2022] DELETE /identity/v3/users/00b673d5c65d40d9a90cc3b579e83e7f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:43.045148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b454b-0033-46c1-810d-e645e34c3b32 service placement] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:43.058292 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9b454b-0033-46c1-810d-e645e34c3b32 service placement] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:43.090699 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:43.117074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89520a2-3717-4078-8cae-f4d84e0021e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:43.117739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89520a2-3717-4078-8cae-f4d84e0021e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:43.118013 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89520a2-3717-4078-8cae-f4d84e0021e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:43.118282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89520a2-3717-4078-8cae-f4d84e0021e6 admin admin] PATH_INFO: `/v3/projects/91730a2601824af19a540c48681e6fc3` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:43.119392 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89520a2-3717-4078-8cae-f4d84e0021e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91730a2601824af19a540c48681e6fc3)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:43.121436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89520a2-3717-4078-8cae-f4d84e0021e6 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:43.174297 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b89520a2-3717-4078-8cae-f4d84e0021e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91730a2601824af19a540c48681e6fc3', 'request_id': 'req-b89520a2-3717-4078-8cae-f4d84e0021e6'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:43.185221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b89520a2-3717-4078-8cae-f4d84e0021e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91730a2601824af19a540c48681e6fc3', 'request_id': 'req-b89520a2-3717-4078-8cae-f4d84e0021e6'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:43.191982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b89520a2-3717-4078-8cae-f4d84e0021e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91730a2601824af19a540c48681e6fc3', 'request_id': 'req-b89520a2-3717-4078-8cae-f4d84e0021e6'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:43.203386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b89520a2-3717-4078-8cae-f4d84e0021e6 admin admin] The token cache is being invalidate because project 91730a2601824af19a540c48681e6fc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:14:43.203705 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b89520a2-3717-4078-8cae-f4d84e0021e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:43.204365 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b89520a2-3717-4078-8cae-f4d84e0021e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:43.205880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1249/2501] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:14:43 2022] DELETE /identity/v3/projects/91730a2601824af19a540c48681e6fc3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:43.211060 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd79a0-722b-4e5a-9341-e756b87af9a5 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:43.211367 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd79a0-722b-4e5a-9341-e756b87af9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:43.212254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd79a0-722b-4e5a-9341-e756b87af9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:43.232866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-85fd79a0-722b-4e5a-9341-e756b87af9a5 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:43.234060 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-85fd79a0-722b-4e5a-9341-e756b87af9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:43.243694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:43.300154 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1253/2502] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:43.308532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13895104-3e28-4fc3-adea-83eb377884d7 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:43.308942 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13895104-3e28-4fc3-adea-83eb377884d7 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:43.309244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13895104-3e28-4fc3-adea-83eb377884d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:43.331535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-13895104-3e28-4fc3-adea-83eb377884d7 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:43.332759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-13895104-3e28-4fc3-adea-83eb377884d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:43.341155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1250/2503] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:43.348931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd79a0-722b-4e5a-9341-e756b87af9a5 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:43.358958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd79a0-722b-4e5a-9341-e756b87af9a5 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:43.363630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878dc354-4fb5-44f4-a287-9060ef242394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:43.364276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878dc354-4fb5-44f4-a287-9060ef242394 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:43.364554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878dc354-4fb5-44f4-a287-9060ef242394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:43.364858 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878dc354-4fb5-44f4-a287-9060ef242394 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:43.365524 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878dc354-4fb5-44f4-a287-9060ef242394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:14:43.365856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878dc354-4fb5-44f4-a287-9060ef242394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:43.367516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878dc354-4fb5-44f4-a287-9060ef242394 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:43.379721 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1254/2504] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:14:43.384259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13895104-3e28-4fc3-adea-83eb377884d7 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:43.394140 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13895104-3e28-4fc3-adea-83eb377884d7 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:43.399587 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b26418-1a7d-48f4-bf4d-512af0b75198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:43.400329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b26418-1a7d-48f4-bf4d-512af0b75198 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:43.400732 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b26418-1a7d-48f4-bf4d-512af0b75198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:43.401182 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b26418-1a7d-48f4-bf4d-512af0b75198 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:43.401965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b26418-1a7d-48f4-bf4d-512af0b75198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:43.404008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b26418-1a7d-48f4-bf4d-512af0b75198 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:43.437135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1251/2505] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:14:43 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:14:43.438248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878dc354-4fb5-44f4-a287-9060ef242394 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:43.449004 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878dc354-4fb5-44f4-a287-9060ef242394 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:43.506210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72587812-b457-401b-9e2b-5818fbc6529b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:43.506813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72587812-b457-401b-9e2b-5818fbc6529b admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:43.507090 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72587812-b457-401b-9e2b-5818fbc6529b admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:43.507471 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72587812-b457-401b-9e2b-5818fbc6529b admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:43.508123 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72587812-b457-401b-9e2b-5818fbc6529b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:43.510031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72587812-b457-401b-9e2b-5818fbc6529b admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:43.530806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1255/2506] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:14:43 2022] POST /identity/v3/users => generated 302 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:14:43.535109 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b26418-1a7d-48f4-bf4d-512af0b75198 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:43.545869 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b26418-1a7d-48f4-bf4d-512af0b75198 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:43.551937 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afff70-ff88-484b-ab1b-49b4774c7326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:43.552534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afff70-ff88-484b-ab1b-49b4774c7326 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:43.552810 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afff70-ff88-484b-ab1b-49b4774c7326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:43.553073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afff70-ff88-484b-ab1b-49b4774c7326 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:43.553716 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afff70-ff88-484b-ab1b-49b4774c7326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:43.555339 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afff70-ff88-484b-ab1b-49b4774c7326 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:43.566240 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1252/2507] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:14:43.570300 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72587812-b457-401b-9e2b-5818fbc6529b admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:43.580753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72587812-b457-401b-9e2b-5818fbc6529b admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:43.586675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659489d4-1036-4a22-81fb-f25b7e771ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:43.587265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659489d4-1036-4a22-81fb-f25b7e771ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:43.587569 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659489d4-1036-4a22-81fb-f25b7e771ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:43.587832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659489d4-1036-4a22-81fb-f25b7e771ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:43.588473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659489d4-1036-4a22-81fb-f25b7e771ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:43.590062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659489d4-1036-4a22-81fb-f25b7e771ad8 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:43.601476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1256/2508] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:14:43.605728 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afff70-ff88-484b-ab1b-49b4774c7326 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:43.616319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afff70-ff88-484b-ab1b-49b4774c7326 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:43.622255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd5b5c-f470-426c-ab68-4fcb47d671cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:43.622902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd5b5c-f470-426c-ab68-4fcb47d671cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:43.623177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd5b5c-f470-426c-ab68-4fcb47d671cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:43.623467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdd5b5c-f470-426c-ab68-4fcb47d671cf admin admin] PATH_INFO: `/v3/projects/ad74c9b0d9804763b2caa4d122eea941/users/31ec6b2e155c435f8ae3628e9cadfc6a/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:43.639222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd5b5c-f470-426c-ab68-4fcb47d671cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad74c9b0d9804763b2caa4d122eea941, user_id=31ec6b2e155c435f8ae3628e9cadfc6a, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:43.641985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdd5b5c-f470-426c-ab68-4fcb47d671cf admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:43.657809 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1253/2509] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:14:43 2022] PUT /identity/v3/projects/ad74c9b0d9804763b2caa4d122eea941/users/31ec6b2e155c435f8ae3628e9cadfc6a/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:43.668425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caea23-4c0e-43c6-9681-2c9856240512 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:43.668939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caea23-4c0e-43c6-9681-2c9856240512 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:43.668939 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44caea23-4c0e-43c6-9681-2c9856240512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:43.699521 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-44caea23-4c0e-43c6-9681-2c9856240512 None None] MFA Rules not processed for user `31ec6b2e155c435f8ae3628e9cadfc6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:43.700650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-44caea23-4c0e-43c6-9681-2c9856240512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:43.766884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1257/2510] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:14:43 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:43.771465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd5b5c-f470-426c-ab68-4fcb47d671cf admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:43.834871 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdd5b5c-f470-426c-ab68-4fcb47d671cf admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:43.840133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a9658-bbe4-42b0-9f61-45f9e23aaefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:43.840764 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a9658-bbe4-42b0-9f61-45f9e23aaefd admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:43.841037 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a9658-bbe4-42b0-9f61-45f9e23aaefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:43.841308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a9658-bbe4-42b0-9f61-45f9e23aaefd admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:43.841931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a9658-bbe4-42b0-9f61-45f9e23aaefd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:43.843892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a9658-bbe4-42b0-9f61-45f9e23aaefd admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:43.849495 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:43.876016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1254/2511] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:14:43 2022] POST /identity/v3/projects => generated 403 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:14:43.879972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44caea23-4c0e-43c6-9681-2c9856240512 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:43.890687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44caea23-4c0e-43c6-9681-2c9856240512 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:43.898659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:43.950077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47bda15-6ca3-43f5-a740-34b7f2a44c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:43.950606 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bda15-6ca3-43f5-a740-34b7f2a44c63 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:43.950881 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bda15-6ca3-43f5-a740-34b7f2a44c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:43.951154 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bda15-6ca3-43f5-a740-34b7f2a44c63 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:43.951818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bda15-6ca3-43f5-a740-34b7f2a44c63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:43.953734 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bda15-6ca3-43f5-a740-34b7f2a44c63 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:43.980302 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1258/2512] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:14:43 2022] POST /identity/v3/users => generated 308 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:14:43.981687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a9658-bbe4-42b0-9f61-45f9e23aaefd admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:43.992728 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a9658-bbe4-42b0-9f61-45f9e23aaefd admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:43.999052 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efd794-feaf-4c3f-8e20-371cab006415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:44.001160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efd794-feaf-4c3f-8e20-371cab006415 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:44.001160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efd794-feaf-4c3f-8e20-371cab006415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:44.001160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efd794-feaf-4c3f-8e20-371cab006415 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:44.001160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efd794-feaf-4c3f-8e20-371cab006415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:44.017677 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efd794-feaf-4c3f-8e20-371cab006415 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:44.039915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1255/2513] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:14:44.044144 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47bda15-6ca3-43f5-a740-34b7f2a44c63 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:44.054489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47bda15-6ca3-43f5-a740-34b7f2a44c63 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:44.059058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec2693-e96f-4d46-8056-ae9a9bfbbb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:44.059732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec2693-e96f-4d46-8056-ae9a9bfbbb89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:44.060009 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec2693-e96f-4d46-8056-ae9a9bfbbb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:44.060277 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec2693-e96f-4d46-8056-ae9a9bfbbb89 admin admin] PATH_INFO: `/v3/projects/8818a02d27d14106a37fe87eab5cb6dc/users/5bca66a28b8743faa30819ec49d53311/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:44.075276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec2693-e96f-4d46-8056-ae9a9bfbbb89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8818a02d27d14106a37fe87eab5cb6dc, user_id=5bca66a28b8743faa30819ec49d53311, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:44.078006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec2693-e96f-4d46-8056-ae9a9bfbbb89 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:44.094446 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1259/2514] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:14:44 2022] PUT /identity/v3/projects/8818a02d27d14106a37fe87eab5cb6dc/users/5bca66a28b8743faa30819ec49d53311/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:44.098723 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efd794-feaf-4c3f-8e20-371cab006415 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:44.109512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efd794-feaf-4c3f-8e20-371cab006415 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:44.169867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a8db3-8f78-4ba7-a106-58b97bb5aa15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:44.170485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a8db3-8f78-4ba7-a106-58b97bb5aa15 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:44.170753 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a8db3-8f78-4ba7-a106-58b97bb5aa15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:44.171057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a8db3-8f78-4ba7-a106-58b97bb5aa15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:44.171851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a8db3-8f78-4ba7-a106-58b97bb5aa15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:44.173467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a8db3-8f78-4ba7-a106-58b97bb5aa15 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:44.184902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1256/2515] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:14:44.189753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec2693-e96f-4d46-8056-ae9a9bfbbb89 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:44.200771 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec2693-e96f-4d46-8056-ae9a9bfbbb89 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:44.205925 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a22ff-2479-45be-8827-ab28a61cc9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:44.206587 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a22ff-2479-45be-8827-ab28a61cc9f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:44.206866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a22ff-2479-45be-8827-ab28a61cc9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:44.207133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a22ff-2479-45be-8827-ab28a61cc9f5 admin admin] PATH_INFO: `/v3/projects/8818a02d27d14106a37fe87eab5cb6dc/users/5bca66a28b8743faa30819ec49d53311/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:44.209199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a22ff-2479-45be-8827-ab28a61cc9f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8818a02d27d14106a37fe87eab5cb6dc, user_id=5bca66a28b8743faa30819ec49d53311, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:44.211897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a22ff-2479-45be-8827-ab28a61cc9f5 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:44.222621 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1260/2516] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:14:44 2022] PUT /identity/v3/projects/8818a02d27d14106a37fe87eab5cb6dc/users/5bca66a28b8743faa30819ec49d53311/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:44.234361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d399f-fbed-497c-a156-9f405878a7d2 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:44.234646 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d399f-fbed-497c-a156-9f405878a7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:44.234920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d399f-fbed-497c-a156-9f405878a7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:44.264903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-418d399f-fbed-497c-a156-9f405878a7d2 None None] MFA Rules not processed for user `5bca66a28b8743faa30819ec49d53311`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:44.266144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-418d399f-fbed-497c-a156-9f405878a7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:44.338354 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1257/2517] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:14:44 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:44.349755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a22ff-2479-45be-8827-ab28a61cc9f5 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:44.351617 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-216a22ff-2479-45be-8827-ab28a61cc9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:44.410265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a22ff-2479-45be-8827-ab28a61cc9f5 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:44.414149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b498b-0c17-4d72-a6ad-c5d787ae7865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:44.414815 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b498b-0c17-4d72-a6ad-c5d787ae7865 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:44.415094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b498b-0c17-4d72-a6ad-c5d787ae7865 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:44.415357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b498b-0c17-4d72-a6ad-c5d787ae7865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:44.424900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b498b-0c17-4d72-a6ad-c5d787ae7865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:44.426332 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b498b-0c17-4d72-a6ad-c5d787ae7865 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:44.436699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1261/2518] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:14:44 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:14:44.575932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d399f-fbed-497c-a156-9f405878a7d2 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:44.634238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:44.641562 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d399f-fbed-497c-a156-9f405878a7d2 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:44.647104 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8aba0-c171-46bb-817a-20bc0dfb9a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:44.647888 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8aba0-c171-46bb-817a-20bc0dfb9a75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:44.648201 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8aba0-c171-46bb-817a-20bc0dfb9a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:44.648470 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8aba0-c171-46bb-817a-20bc0dfb9a75 admin admin] PATH_INFO: `/v3/users/31ec6b2e155c435f8ae3628e9cadfc6a` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:44.649619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8aba0-c171-46bb-817a-20bc0dfb9a75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31ec6b2e155c435f8ae3628e9cadfc6a)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:44.651668 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b8aba0-c171-46bb-817a-20bc0dfb9a75 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:44.701209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-54b8aba0-c171-46bb-817a-20bc0dfb9a75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31ec6b2e155c435f8ae3628e9cadfc6a'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:44.709693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-54b8aba0-c171-46bb-817a-20bc0dfb9a75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31ec6b2e155c435f8ae3628e9cadfc6a'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:44.710331 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-54b8aba0-c171-46bb-817a-20bc0dfb9a75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31ec6b2e155c435f8ae3628e9cadfc6a'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:44.710954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-54b8aba0-c171-46bb-817a-20bc0dfb9a75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31ec6b2e155c435f8ae3628e9cadfc6a'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:44.723071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-54b8aba0-c171-46bb-817a-20bc0dfb9a75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31ec6b2e155c435f8ae3628e9cadfc6a'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:44.735227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1258/2519] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:14:44 2022] DELETE /identity/v3/users/31ec6b2e155c435f8ae3628e9cadfc6a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:44.739979 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a126c74-fa79-45d3-8736-cee962de843e None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:44.740475 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a126c74-fa79-45d3-8736-cee962de843e None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:44.740572 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a126c74-fa79-45d3-8736-cee962de843e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:44.760284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-3a126c74-fa79-45d3-8736-cee962de843e None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:44.761518 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3a126c74-fa79-45d3-8736-cee962de843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:44.822016 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1262/2520] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:44.832180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8aba0-c171-46bb-817a-20bc0dfb9a75 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:44.833718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-54b8aba0-c171-46bb-817a-20bc0dfb9a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:44.898750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8aba0-c171-46bb-817a-20bc0dfb9a75 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:44.902517 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a5d21-6067-4112-832b-9f79a7bc74c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:44.903176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a5d21-6067-4112-832b-9f79a7bc74c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:44.903467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a5d21-6067-4112-832b-9f79a7bc74c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:44.903741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a5d21-6067-4112-832b-9f79a7bc74c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:44.913032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a5d21-6067-4112-832b-9f79a7bc74c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:44.914594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a5d21-6067-4112-832b-9f79a7bc74c0 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:44.924224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1259/2521] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:14:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:14:45.209372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a126c74-fa79-45d3-8736-cee962de843e None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:45.211116 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3a126c74-fa79-45d3-8736-cee962de843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:45.225725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a126c74-fa79-45d3-8736-cee962de843e None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:45.233264 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62aa36-a186-4deb-810f-7ea4b95c1341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:45.234108 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62aa36-a186-4deb-810f-7ea4b95c1341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:45.234429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62aa36-a186-4deb-810f-7ea4b95c1341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:45.234947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62aa36-a186-4deb-810f-7ea4b95c1341 admin admin] PATH_INFO: `/v3/projects/ad74c9b0d9804763b2caa4d122eea941` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:45.236463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62aa36-a186-4deb-810f-7ea4b95c1341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad74c9b0d9804763b2caa4d122eea941)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:45.239161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62aa36-a186-4deb-810f-7ea4b95c1341 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:45.241028 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:45.289962 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-dc62aa36-a186-4deb-810f-7ea4b95c1341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad74c9b0d9804763b2caa4d122eea941', 'request_id': 'req-dc62aa36-a186-4deb-810f-7ea4b95c1341'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:45.298902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-dc62aa36-a186-4deb-810f-7ea4b95c1341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad74c9b0d9804763b2caa4d122eea941', 'request_id': 'req-dc62aa36-a186-4deb-810f-7ea4b95c1341'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:45.305675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-dc62aa36-a186-4deb-810f-7ea4b95c1341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad74c9b0d9804763b2caa4d122eea941', 'request_id': 'req-dc62aa36-a186-4deb-810f-7ea4b95c1341'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:45.318341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-dc62aa36-a186-4deb-810f-7ea4b95c1341 admin admin] The token cache is being invalidate because project ad74c9b0d9804763b2caa4d122eea941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:14:45.318642 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-dc62aa36-a186-4deb-810f-7ea4b95c1341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:45.319475 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-dc62aa36-a186-4deb-810f-7ea4b95c1341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:45.321063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1263/2522] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:14:45 2022] DELETE /identity/v3/projects/ad74c9b0d9804763b2caa4d122eea941 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:45.325291 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a5d21-6067-4112-832b-9f79a7bc74c0 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:45.335779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a5d21-6067-4112-832b-9f79a7bc74c0 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:45.397212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89d8b9-9a97-468e-b6f7-744fb8cd8ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:45.397867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89d8b9-9a97-468e-b6f7-744fb8cd8ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:45.398191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89d8b9-9a97-468e-b6f7-744fb8cd8ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:45.398483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89d8b9-9a97-468e-b6f7-744fb8cd8ff9 admin admin] PATH_INFO: `/v3/users/5bca66a28b8743faa30819ec49d53311` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:45.399689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89d8b9-9a97-468e-b6f7-744fb8cd8ff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bca66a28b8743faa30819ec49d53311)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:45.401656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89d8b9-9a97-468e-b6f7-744fb8cd8ff9 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:45.424444 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:45.453355 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0b89d8b9-9a97-468e-b6f7-744fb8cd8ff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bca66a28b8743faa30819ec49d53311'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:45.462214 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0b89d8b9-9a97-468e-b6f7-744fb8cd8ff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bca66a28b8743faa30819ec49d53311'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:45.462958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0b89d8b9-9a97-468e-b6f7-744fb8cd8ff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bca66a28b8743faa30819ec49d53311'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:45.463587 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0b89d8b9-9a97-468e-b6f7-744fb8cd8ff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bca66a28b8743faa30819ec49d53311'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:45.476174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0b89d8b9-9a97-468e-b6f7-744fb8cd8ff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bca66a28b8743faa30819ec49d53311'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:45.488758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1260/2523] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:14:45 2022] DELETE /identity/v3/users/5bca66a28b8743faa30819ec49d53311 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:45.788233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62aa36-a186-4deb-810f-7ea4b95c1341 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:45.789872 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-dc62aa36-a186-4deb-810f-7ea4b95c1341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:45.802718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62aa36-a186-4deb-810f-7ea4b95c1341 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:45.860289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d104035-9463-4295-9c87-c3030b12ed2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:45.860920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d104035-9463-4295-9c87-c3030b12ed2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:45.861189 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d104035-9463-4295-9c87-c3030b12ed2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:45.861461 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d104035-9463-4295-9c87-c3030b12ed2c admin admin] PATH_INFO: `/v3/projects/8818a02d27d14106a37fe87eab5cb6dc` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:45.862578 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d104035-9463-4295-9c87-c3030b12ed2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8818a02d27d14106a37fe87eab5cb6dc)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:45.864524 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d104035-9463-4295-9c87-c3030b12ed2c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:45.913185 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8d104035-9463-4295-9c87-c3030b12ed2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8818a02d27d14106a37fe87eab5cb6dc', 'request_id': 'req-8d104035-9463-4295-9c87-c3030b12ed2c'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:45.922545 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8d104035-9463-4295-9c87-c3030b12ed2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8818a02d27d14106a37fe87eab5cb6dc', 'request_id': 'req-8d104035-9463-4295-9c87-c3030b12ed2c'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:45.928295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8d104035-9463-4295-9c87-c3030b12ed2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8818a02d27d14106a37fe87eab5cb6dc', 'request_id': 'req-8d104035-9463-4295-9c87-c3030b12ed2c'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:45.939990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8d104035-9463-4295-9c87-c3030b12ed2c admin admin] The token cache is being invalidate because project 8818a02d27d14106a37fe87eab5cb6dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:14:45.940278 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8d104035-9463-4295-9c87-c3030b12ed2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:45.941010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-8d104035-9463-4295-9c87-c3030b12ed2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:14:45.942613 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1264/2524] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:14:45 2022] DELETE /identity/v3/projects/8818a02d27d14106a37fe87eab5cb6dc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:45.947883 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367b57-9a09-48e7-bc56-fae9614a8f8e None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:45.948360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367b57-9a09-48e7-bc56-fae9614a8f8e None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:45.948455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50367b57-9a09-48e7-bc56-fae9614a8f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:45.969311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-50367b57-9a09-48e7-bc56-fae9614a8f8e None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:45.970509 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-50367b57-9a09-48e7-bc56-fae9614a8f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:46.031012 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1261/2525] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:14:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:14:46.039297 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214b91e-326b-4158-970a-045fff8532cd None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:46.040988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214b91e-326b-4158-970a-045fff8532cd None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:46.040988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8214b91e-326b-4158-970a-045fff8532cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:46.061923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-8214b91e-326b-4158-970a-045fff8532cd None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:46.063132 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-8214b91e-326b-4158-970a-045fff8532cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:46.074139 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1265/2526] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:46.080951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50367b57-9a09-48e7-bc56-fae9614a8f8e None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:46.095903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50367b57-9a09-48e7-bc56-fae9614a8f8e None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:46.103887 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569859f-2980-4f4d-929c-9291f59acd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:46.104757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569859f-2980-4f4d-929c-9291f59acd2f admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:46.105085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569859f-2980-4f4d-929c-9291f59acd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:46.105373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2569859f-2980-4f4d-929c-9291f59acd2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:46.106223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569859f-2980-4f4d-929c-9291f59acd2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:14:46.106528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569859f-2980-4f4d-929c-9291f59acd2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:46.108291 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2569859f-2980-4f4d-929c-9291f59acd2f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:46.120810 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1262/2527] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:14:46.125543 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214b91e-326b-4158-970a-045fff8532cd None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:46.135470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8214b91e-326b-4158-970a-045fff8532cd None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:46.141332 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304bc3f-ed2c-477e-96eb-c7c0d5224717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:46.141954 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304bc3f-ed2c-477e-96eb-c7c0d5224717 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:46.142256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304bc3f-ed2c-477e-96eb-c7c0d5224717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:46.142552 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304bc3f-ed2c-477e-96eb-c7c0d5224717 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:46.143189 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304bc3f-ed2c-477e-96eb-c7c0d5224717 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:46.145175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304bc3f-ed2c-477e-96eb-c7c0d5224717 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:46.157368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:46.179262 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1266/2528] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:14:46 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:14:46.183716 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569859f-2980-4f4d-929c-9291f59acd2f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:46.198873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2569859f-2980-4f4d-929c-9291f59acd2f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:46.262678 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dc8eb-dea6-44e5-b293-85dba7c296d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:46.263331 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dc8eb-dea6-44e5-b293-85dba7c296d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:46.263640 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dc8eb-dea6-44e5-b293-85dba7c296d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:46.263938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dc8eb-dea6-44e5-b293-85dba7c296d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:46.264604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720dc8eb-dea6-44e5-b293-85dba7c296d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:46.266596 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720dc8eb-dea6-44e5-b293-85dba7c296d5 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:46.290410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1263/2529] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:14:46 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:14:46.294796 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304bc3f-ed2c-477e-96eb-c7c0d5224717 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:46.305667 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304bc3f-ed2c-477e-96eb-c7c0d5224717 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:46.311518 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc888f8-265f-426f-a2cb-06cfb450c36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:46.312141 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc888f8-265f-426f-a2cb-06cfb450c36c admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:46.312413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc888f8-265f-426f-a2cb-06cfb450c36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:46.312676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc888f8-265f-426f-a2cb-06cfb450c36c admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:46.313316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc888f8-265f-426f-a2cb-06cfb450c36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:46.314922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc888f8-265f-426f-a2cb-06cfb450c36c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:46.325713 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1267/2530] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:14:46.330010 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dc8eb-dea6-44e5-b293-85dba7c296d5 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:46.340534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dc8eb-dea6-44e5-b293-85dba7c296d5 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:46.346163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f1fb8-5587-4d80-9f66-dcb23fa86934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:46.346762 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f1fb8-5587-4d80-9f66-dcb23fa86934 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:46.347035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f1fb8-5587-4d80-9f66-dcb23fa86934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:46.347298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f1fb8-5587-4d80-9f66-dcb23fa86934 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:46.347964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f1fb8-5587-4d80-9f66-dcb23fa86934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:46.349570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0f1fb8-5587-4d80-9f66-dcb23fa86934 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:46.360775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1264/2531] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:14:46.364991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc888f8-265f-426f-a2cb-06cfb450c36c admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:46.375103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc888f8-265f-426f-a2cb-06cfb450c36c admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:46.380120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f43d54-192f-45bb-8953-482649c9c163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:46.380783 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f43d54-192f-45bb-8953-482649c9c163 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:46.381056 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f43d54-192f-45bb-8953-482649c9c163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:46.381322 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f43d54-192f-45bb-8953-482649c9c163 admin admin] PATH_INFO: `/v3/projects/77729cc5955b43c2acbf736d7169a8cc/users/7729065280d344f28f7a4983ce46bf12/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:46.396271 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f43d54-192f-45bb-8953-482649c9c163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77729cc5955b43c2acbf736d7169a8cc, user_id=7729065280d344f28f7a4983ce46bf12, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:46.398909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f43d54-192f-45bb-8953-482649c9c163 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:46.414418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1268/2532] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:14:46 2022] PUT /identity/v3/projects/77729cc5955b43c2acbf736d7169a8cc/users/7729065280d344f28f7a4983ce46bf12/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:46.425151 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029c22e-3d8e-4e90-8946-4f047264bdb6 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:46.425409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029c22e-3d8e-4e90-8946-4f047264bdb6 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:46.425681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029c22e-3d8e-4e90-8946-4f047264bdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:46.433253 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:46.456852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-1029c22e-3d8e-4e90-8946-4f047264bdb6 None None] MFA Rules not processed for user `7729065280d344f28f7a4983ce46bf12`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:46.458078 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-1029c22e-3d8e-4e90-8946-4f047264bdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:46.527193 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1265/2533] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:14:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:46.533897 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f43d54-192f-45bb-8953-482649c9c163 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:46.610092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f43d54-192f-45bb-8953-482649c9c163 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:46.615240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0bf3b-c2af-4783-8989-3310b30479a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:46.615871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0bf3b-c2af-4783-8989-3310b30479a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:46.616148 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0bf3b-c2af-4783-8989-3310b30479a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:46.616414 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0bf3b-c2af-4783-8989-3310b30479a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:46.617038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0bf3b-c2af-4783-8989-3310b30479a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:46.618915 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0bf3b-c2af-4783-8989-3310b30479a2 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:46.647866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1269/2534] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:14:46 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:14:46.652724 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029c22e-3d8e-4e90-8946-4f047264bdb6 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:46.667811 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029c22e-3d8e-4e90-8946-4f047264bdb6 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:46.724362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e8f04-be24-478f-a641-67dd3e4b1b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:46.724972 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e8f04-be24-478f-a641-67dd3e4b1b26 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:46.725240 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e8f04-be24-478f-a641-67dd3e4b1b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:46.725500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15e8f04-be24-478f-a641-67dd3e4b1b26 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:46.726148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e8f04-be24-478f-a641-67dd3e4b1b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:46.728073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15e8f04-be24-478f-a641-67dd3e4b1b26 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:46.749746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1266/2535] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:14:46 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:14:46.753724 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0bf3b-c2af-4783-8989-3310b30479a2 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:46.764333 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0bf3b-c2af-4783-8989-3310b30479a2 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:46.769884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1dee89-a5d0-4ad2-83ef-f41637383e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:46.770492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dee89-a5d0-4ad2-83ef-f41637383e49 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:46.770762 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dee89-a5d0-4ad2-83ef-f41637383e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:46.771024 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dee89-a5d0-4ad2-83ef-f41637383e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:46.771736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dee89-a5d0-4ad2-83ef-f41637383e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:46.773413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dee89-a5d0-4ad2-83ef-f41637383e49 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:46.784308 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1270/2536] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:14:46.788690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e8f04-be24-478f-a641-67dd3e4b1b26 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:46.798919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15e8f04-be24-478f-a641-67dd3e4b1b26 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:46.803721 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c271a1-4fa7-4246-b0b2-b828e531b572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:46.804435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c271a1-4fa7-4246-b0b2-b828e531b572 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:46.804707 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c271a1-4fa7-4246-b0b2-b828e531b572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:46.805000 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c271a1-4fa7-4246-b0b2-b828e531b572 admin admin] PATH_INFO: `/v3/projects/8b2f5de189d34303977d0fdf18e9a9b9/users/d02564aad59245af9a5d404e2eacdc56/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:46.821795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c271a1-4fa7-4246-b0b2-b828e531b572 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2f5de189d34303977d0fdf18e9a9b9, user_id=d02564aad59245af9a5d404e2eacdc56, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:46.824614 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c271a1-4fa7-4246-b0b2-b828e531b572 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:46.839970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1267/2537] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:14:46 2022] PUT /identity/v3/projects/8b2f5de189d34303977d0fdf18e9a9b9/users/d02564aad59245af9a5d404e2eacdc56/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:46.844108 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1dee89-a5d0-4ad2-83ef-f41637383e49 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:46.854409 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1dee89-a5d0-4ad2-83ef-f41637383e49 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:46.913645 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b426c3-fa4d-46fd-ba4d-e60a3402b587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:46.914336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b426c3-fa4d-46fd-ba4d-e60a3402b587 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:46.914679 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b426c3-fa4d-46fd-ba4d-e60a3402b587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:46.915009 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b426c3-fa4d-46fd-ba4d-e60a3402b587 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:46.915777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b426c3-fa4d-46fd-ba4d-e60a3402b587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:46.917457 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b426c3-fa4d-46fd-ba4d-e60a3402b587 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:46.927504 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1271/2538] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:14:46.931869 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c271a1-4fa7-4246-b0b2-b828e531b572 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:46.942578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c271a1-4fa7-4246-b0b2-b828e531b572 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:46.947382 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8849dd7d-b4d5-478e-92f4-5140d9eefdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:46.948072 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849dd7d-b4d5-478e-92f4-5140d9eefdb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:46.948624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849dd7d-b4d5-478e-92f4-5140d9eefdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:46.948624 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8849dd7d-b4d5-478e-92f4-5140d9eefdb7 admin admin] PATH_INFO: `/v3/projects/8b2f5de189d34303977d0fdf18e9a9b9/users/d02564aad59245af9a5d404e2eacdc56/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:46.950596 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8849dd7d-b4d5-478e-92f4-5140d9eefdb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b2f5de189d34303977d0fdf18e9a9b9, user_id=d02564aad59245af9a5d404e2eacdc56, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:46.953280 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8849dd7d-b4d5-478e-92f4-5140d9eefdb7 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:46.963873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1268/2539] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:14:46 2022] PUT /identity/v3/projects/8b2f5de189d34303977d0fdf18e9a9b9/users/d02564aad59245af9a5d404e2eacdc56/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:14:46.981687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950795ae-63c7-4ffc-9b44-78e673950a19 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:46.982023 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950795ae-63c7-4ffc-9b44-78e673950a19 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:46.982389 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950795ae-63c7-4ffc-9b44-78e673950a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:47.014010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-950795ae-63c7-4ffc-9b44-78e673950a19 None None] MFA Rules not processed for user `d02564aad59245af9a5d404e2eacdc56`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:47.015212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-950795ae-63c7-4ffc-9b44-78e673950a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:47.029506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:47.088581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1272/2540] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:14:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:47.104088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8849dd7d-b4d5-478e-92f4-5140d9eefdb7 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:47.105833 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8849dd7d-b4d5-478e-92f4-5140d9eefdb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:47.156472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:14:47.170020 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8849dd7d-b4d5-478e-92f4-5140d9eefdb7 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:47.174261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a2beb-b61b-41ee-a719-b27ab79f30b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:47.174881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a2beb-b61b-41ee-a719-b27ab79f30b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:47.175157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a2beb-b61b-41ee-a719-b27ab79f30b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:47.175425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a2beb-b61b-41ee-a719-b27ab79f30b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:47.241891 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a2beb-b61b-41ee-a719-b27ab79f30b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:47.243337 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a2beb-b61b-41ee-a719-b27ab79f30b6 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:47.252144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1269/2541] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:14:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:14:48.263231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812737b0-c770-4026-ba76-5ea1cd55ea67 None None] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:48.263802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812737b0-c770-4026-ba76-5ea1cd55ea67 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:48.263802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812737b0-c770-4026-ba76-5ea1cd55ea67 None None] PATH_INFO: `/` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:48.265203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1273/2542] 10.222.0.27 () {54 vars in 921 bytes} [Thu Sep 8 16:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:14:48.271408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ce214-5195-4117-a06f-b192cae6be58 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:48.271963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ce214-5195-4117-a06f-b192cae6be58 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:48.271963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ce214-5195-4117-a06f-b192cae6be58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:48.302988 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-831ce214-5195-4117-a06f-b192cae6be58 None None] MFA Rules not processed for user `3aae65f9ec394f97aeafad43932dc49c`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:48.304375 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-831ce214-5195-4117-a06f-b192cae6be58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:48.356362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1270/2543] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Sep 8 16:14:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:14:48.368466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812737b0-c770-4026-ba76-5ea1cd55ea67 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:48.370295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-812737b0-c770-4026-ba76-5ea1cd55ea67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:48.384285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812737b0-c770-4026-ba76-5ea1cd55ea67 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:48.388218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb564fd9-9a89-4f71-ad98-853c3ab9693a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:48.388856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb564fd9-9a89-4f71-ad98-853c3ab9693a service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:48.389135 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb564fd9-9a89-4f71-ad98-853c3ab9693a service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:48.389407 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb564fd9-9a89-4f71-ad98-853c3ab9693a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:48.400382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb564fd9-9a89-4f71-ad98-853c3ab9693a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:48.401843 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb564fd9-9a89-4f71-ad98-853c3ab9693a service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:48.409904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1274/2544] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:14:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:14:48.494823 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d1666-a756-428b-bf84-c8be86349c14 None None] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:48.495073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d1666-a756-428b-bf84-c8be86349c14 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:48.495340 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d1666-a756-428b-bf84-c8be86349c14 None None] PATH_INFO: `/` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:48.496754 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1271/2545] 10.222.0.27 () {54 vars in 921 bytes} [Thu Sep 8 16:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 16:14:48.502277 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f90c07-5dcc-43b0-bfd2-63821f4224dc None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:48.502563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f90c07-5dcc-43b0-bfd2-63821f4224dc None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:48.502827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f90c07-5dcc-43b0-bfd2-63821f4224dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:48.527900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-d3f90c07-5dcc-43b0-bfd2-63821f4224dc None None] MFA Rules not processed for user `6e9209e77ce44b128e25a8ddcf78f8b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:14:48.529117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d3f90c07-5dcc-43b0-bfd2-63821f4224dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:48.603875 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1275/2546] 10.222.0.27 () {58 vars in 1018 bytes} [Thu Sep 8 16:14:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:14:48.614662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d1666-a756-428b-bf84-c8be86349c14 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:48.616473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-162d1666-a756-428b-bf84-c8be86349c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:14:48.630375 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d1666-a756-428b-bf84-c8be86349c14 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:48.634417 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f3c85-f2bf-456f-8b24-d1d1d7a7a4d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:48.635116 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f3c85-f2bf-456f-8b24-d1d1d7a7a4d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:48.635490 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f3c85-f2bf-456f-8b24-d1d1d7a7a4d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:48.635766 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f3c85-f2bf-456f-8b24-d1d1d7a7a4d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:48.646470 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f3c85-f2bf-456f-8b24-d1d1d7a7a4d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:48.647850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f3c85-f2bf-456f-8b24-d1d1d7a7a4d1 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:48.658831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1272/2547] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:14:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:14:51.910200 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f90c07-5dcc-43b0-bfd2-63821f4224dc None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:14:51.920723 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f90c07-5dcc-43b0-bfd2-63821f4224dc None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:14:51.932236 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbff72d-df53-4934-9b62-db3c1e405cad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:14:51.933588 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbff72d-df53-4934-9b62-db3c1e405cad service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:14:51.934061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbff72d-df53-4934-9b62-db3c1e405cad service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:14:51.935292 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbff72d-df53-4934-9b62-db3c1e405cad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:14:51.950660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbff72d-df53-4934-9b62-db3c1e405cad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:14:51.952888 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbff72d-df53-4934-9b62-db3c1e405cad service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:14:51.965353 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1276/2548] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:14:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:14:58.654751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=57 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:01.960898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=65 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:13.681655 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f3c85-f2bf-456f-8b24-d1d1d7a7a4d1 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:13.691189 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f3c85-f2bf-456f-8b24-d1d1d7a7a4d1 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:13.695689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f5472e-f80b-4b8d-b454-15ceabebbace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:13.696285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5472e-f80b-4b8d-b454-15ceabebbace service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:13.696556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5472e-f80b-4b8d-b454-15ceabebbace service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:13.696815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f5472e-f80b-4b8d-b454-15ceabebbace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:13.698929 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-76f5472e-f80b-4b8d-b454-15ceabebbace service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:13.763256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f5472e-f80b-4b8d-b454-15ceabebbace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:13.764743 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f5472e-f80b-4b8d-b454-15ceabebbace service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:13.774444 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1273/2549] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:14.975682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbff72d-df53-4934-9b62-db3c1e405cad service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:14.986455 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbff72d-df53-4934-9b62-db3c1e405cad service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:14.991282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09dc9ee-9866-4433-85e8-a786a064f447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:14.991966 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dc9ee-9866-4433-85e8-a786a064f447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:14.992243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dc9ee-9866-4433-85e8-a786a064f447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:14.992510 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09dc9ee-9866-4433-85e8-a786a064f447 admin admin] PATH_INFO: `/v3/users/7729065280d344f28f7a4983ce46bf12` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:14.993842 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09dc9ee-9866-4433-85e8-a786a064f447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7729065280d344f28f7a4983ce46bf12)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:14.996102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09dc9ee-9866-4433-85e8-a786a064f447 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:15.050813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a09dc9ee-9866-4433-85e8-a786a064f447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7729065280d344f28f7a4983ce46bf12'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:15.060636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a09dc9ee-9866-4433-85e8-a786a064f447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7729065280d344f28f7a4983ce46bf12'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:15.061584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a09dc9ee-9866-4433-85e8-a786a064f447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7729065280d344f28f7a4983ce46bf12'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:15.062392 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a09dc9ee-9866-4433-85e8-a786a064f447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7729065280d344f28f7a4983ce46bf12'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:15.073589 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a09dc9ee-9866-4433-85e8-a786a064f447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7729065280d344f28f7a4983ce46bf12'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:15.098709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1277/2550] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:14 2022] DELETE /identity/v3/users/7729065280d344f28f7a4983ce46bf12 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:15.103141 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c3280-89a1-4d8f-b53a-ff09918fb211 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:15.103643 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c3280-89a1-4d8f-b53a-ff09918fb211 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:15.103746 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c3280-89a1-4d8f-b53a-ff09918fb211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:15.125004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-602c3280-89a1-4d8f-b53a-ff09918fb211 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:15.126362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-602c3280-89a1-4d8f-b53a-ff09918fb211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:15.193488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1274/2551] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:15.204066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09dc9ee-9866-4433-85e8-a786a064f447 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:15.205782 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a09dc9ee-9866-4433-85e8-a786a064f447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:15.263570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09dc9ee-9866-4433-85e8-a786a064f447 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:15.267293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841165a-6451-4640-a9e0-c86234912247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:15.267937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841165a-6451-4640-a9e0-c86234912247 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:15.268228 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841165a-6451-4640-a9e0-c86234912247 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:15.268481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841165a-6451-4640-a9e0-c86234912247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:15.278654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841165a-6451-4640-a9e0-c86234912247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:15.280184 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841165a-6451-4640-a9e0-c86234912247 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:15.290051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1278/2552] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:15.588322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c3280-89a1-4d8f-b53a-ff09918fb211 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:15.590022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-602c3280-89a1-4d8f-b53a-ff09918fb211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:15.603172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c3280-89a1-4d8f-b53a-ff09918fb211 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:15.609603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:15.610257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:15.610522 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:15.610786 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568 admin admin] PATH_INFO: `/v3/projects/77729cc5955b43c2acbf736d7169a8cc` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:15.611972 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77729cc5955b43c2acbf736d7169a8cc)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:15.613938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:15.679408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77729cc5955b43c2acbf736d7169a8cc', 'request_id': 'req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:15.690314 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77729cc5955b43c2acbf736d7169a8cc', 'request_id': 'req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:15.697361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77729cc5955b43c2acbf736d7169a8cc', 'request_id': 'req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:15.710285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568 admin admin] The token cache is being invalidate because project 77729cc5955b43c2acbf736d7169a8cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:15.710502 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:15.711224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:15.713330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1275/2553] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:15 2022] DELETE /identity/v3/projects/77729cc5955b43c2acbf736d7169a8cc => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:15.717337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841165a-6451-4640-a9e0-c86234912247 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:15.728258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841165a-6451-4640-a9e0-c86234912247 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:15.793755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9563c-c168-4f9b-9819-0f75989d11a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:15.794522 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9563c-c168-4f9b-9819-0f75989d11a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:15.794999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9563c-c168-4f9b-9819-0f75989d11a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:15.795424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9563c-c168-4f9b-9819-0f75989d11a2 admin admin] PATH_INFO: `/v3/users/d02564aad59245af9a5d404e2eacdc56` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:15.796736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9563c-c168-4f9b-9819-0f75989d11a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02564aad59245af9a5d404e2eacdc56)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:15.798936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9563c-c168-4f9b-9819-0f75989d11a2 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:15.818427 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:15.857029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-34e9563c-c168-4f9b-9819-0f75989d11a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02564aad59245af9a5d404e2eacdc56'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:15.866253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-34e9563c-c168-4f9b-9819-0f75989d11a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02564aad59245af9a5d404e2eacdc56'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:15.866921 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-34e9563c-c168-4f9b-9819-0f75989d11a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02564aad59245af9a5d404e2eacdc56'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:15.867583 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-34e9563c-c168-4f9b-9819-0f75989d11a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02564aad59245af9a5d404e2eacdc56'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:15.875694 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-34e9563c-c168-4f9b-9819-0f75989d11a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02564aad59245af9a5d404e2eacdc56'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:15.894670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1279/2554] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:15 2022] DELETE /identity/v3/users/d02564aad59245af9a5d404e2eacdc56 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:16.193770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:16.195785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:16.211875 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42686e-4d6d-4f78-bf2b-9aaaebdd9568 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:16.227865 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:16.276885 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f180c-cb96-4efa-9517-c00e8f4556ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:16.277532 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f180c-cb96-4efa-9517-c00e8f4556ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:16.277813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f180c-cb96-4efa-9517-c00e8f4556ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:16.278113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f180c-cb96-4efa-9517-c00e8f4556ad admin admin] PATH_INFO: `/v3/projects/8b2f5de189d34303977d0fdf18e9a9b9` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:16.279224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f180c-cb96-4efa-9517-c00e8f4556ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b2f5de189d34303977d0fdf18e9a9b9)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:16.281316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92f180c-cb96-4efa-9517-c00e8f4556ad admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:16.329424 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e92f180c-cb96-4efa-9517-c00e8f4556ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b2f5de189d34303977d0fdf18e9a9b9', 'request_id': 'req-e92f180c-cb96-4efa-9517-c00e8f4556ad'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:16.338100 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e92f180c-cb96-4efa-9517-c00e8f4556ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b2f5de189d34303977d0fdf18e9a9b9', 'request_id': 'req-e92f180c-cb96-4efa-9517-c00e8f4556ad'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:16.343829 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e92f180c-cb96-4efa-9517-c00e8f4556ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b2f5de189d34303977d0fdf18e9a9b9', 'request_id': 'req-e92f180c-cb96-4efa-9517-c00e8f4556ad'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:16.355989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e92f180c-cb96-4efa-9517-c00e8f4556ad admin admin] The token cache is being invalidate because project 8b2f5de189d34303977d0fdf18e9a9b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:16.356402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e92f180c-cb96-4efa-9517-c00e8f4556ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:16.357183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e92f180c-cb96-4efa-9517-c00e8f4556ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:16.359077 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1276/2555] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:16 2022] DELETE /identity/v3/projects/8b2f5de189d34303977d0fdf18e9a9b9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:16.363995 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca1e6a-8412-40ab-8069-5e5645dfc073 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:16.364497 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca1e6a-8412-40ab-8069-5e5645dfc073 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:16.364609 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca1e6a-8412-40ab-8069-5e5645dfc073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:16.384816 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-3dca1e6a-8412-40ab-8069-5e5645dfc073 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:16.386043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3dca1e6a-8412-40ab-8069-5e5645dfc073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:16.454299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1280/2556] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:16.462409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9b67b-886d-4e59-884a-99578dffdb52 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:16.462714 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9b67b-886d-4e59-884a-99578dffdb52 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:16.462990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9b67b-886d-4e59-884a-99578dffdb52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:16.489921 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-60a9b67b-886d-4e59-884a-99578dffdb52 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:16.491189 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-60a9b67b-886d-4e59-884a-99578dffdb52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:16.502663 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1277/2557] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:16.506764 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca1e6a-8412-40ab-8069-5e5645dfc073 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:16.521347 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca1e6a-8412-40ab-8069-5e5645dfc073 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:16.531402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc82471-3c24-41c6-882a-cfcb8d4616cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:16.532065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc82471-3c24-41c6-882a-cfcb8d4616cd admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:16.532337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc82471-3c24-41c6-882a-cfcb8d4616cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:16.532607 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc82471-3c24-41c6-882a-cfcb8d4616cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:16.533263 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc82471-3c24-41c6-882a-cfcb8d4616cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:15:16.533561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc82471-3c24-41c6-882a-cfcb8d4616cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:16.535195 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc82471-3c24-41c6-882a-cfcb8d4616cd admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:16.553991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1281/2558] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:15:16.558435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9b67b-886d-4e59-884a-99578dffdb52 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:16.569083 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9b67b-886d-4e59-884a-99578dffdb52 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:16.575574 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd858d8e-e6cb-466b-928b-df56336ae06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:16.576222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd858d8e-e6cb-466b-928b-df56336ae06e admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:16.576530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd858d8e-e6cb-466b-928b-df56336ae06e admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:16.576798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd858d8e-e6cb-466b-928b-df56336ae06e admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:16.577450 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd858d8e-e6cb-466b-928b-df56336ae06e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:16.579373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd858d8e-e6cb-466b-928b-df56336ae06e admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:16.609534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1278/2559] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:16 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:16.613893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc82471-3c24-41c6-882a-cfcb8d4616cd admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:16.624838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc82471-3c24-41c6-882a-cfcb8d4616cd admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:16.822819 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c4815-30c1-4302-8056-7ef90f5a9887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:16.823438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c4815-30c1-4302-8056-7ef90f5a9887 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:16.823825 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c4815-30c1-4302-8056-7ef90f5a9887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:16.824102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c4815-30c1-4302-8056-7ef90f5a9887 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:16.824790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c4815-30c1-4302-8056-7ef90f5a9887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:16.826857 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c4815-30c1-4302-8056-7ef90f5a9887 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:16.863001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1282/2560] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:16 2022] POST /identity/v3/users => generated 287 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:16.863948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd858d8e-e6cb-466b-928b-df56336ae06e admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:16.882607 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd858d8e-e6cb-466b-928b-df56336ae06e admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:16.890267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6418f6-dee3-4257-a648-a5dd46d6e80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:16.891137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6418f6-dee3-4257-a648-a5dd46d6e80e admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:16.891634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6418f6-dee3-4257-a648-a5dd46d6e80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:16.892185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6418f6-dee3-4257-a648-a5dd46d6e80e admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:16.893012 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6418f6-dee3-4257-a648-a5dd46d6e80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:16.895131 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6418f6-dee3-4257-a648-a5dd46d6e80e admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:16.919921 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1279/2561] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:16.924380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c4815-30c1-4302-8056-7ef90f5a9887 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:16.934685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c4815-30c1-4302-8056-7ef90f5a9887 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:16.939995 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce5188-afd9-4388-abf7-ad20188e6fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:16.940606 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce5188-afd9-4388-abf7-ad20188e6fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:16.940884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce5188-afd9-4388-abf7-ad20188e6fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:16.941152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce5188-afd9-4388-abf7-ad20188e6fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:16.941806 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce5188-afd9-4388-abf7-ad20188e6fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:16.943407 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce5188-afd9-4388-abf7-ad20188e6fb5 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:16.949068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:16.955660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1283/2562] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:16.958907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6418f6-dee3-4257-a648-a5dd46d6e80e admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:16.969921 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6418f6-dee3-4257-a648-a5dd46d6e80e admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:16.974903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d55e5-f641-4af8-af30-9cb1205ad401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:16.975620 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d55e5-f641-4af8-af30-9cb1205ad401 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:16.975894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d55e5-f641-4af8-af30-9cb1205ad401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:16.976220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d55e5-f641-4af8-af30-9cb1205ad401 admin admin] PATH_INFO: `/v3/projects/81cbb2a3fa1541eebd180ba990e69838/users/d5be511ce1994a598756d63284a49846/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:16.995242 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d55e5-f641-4af8-af30-9cb1205ad401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81cbb2a3fa1541eebd180ba990e69838, user_id=d5be511ce1994a598756d63284a49846, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:16.998229 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d55e5-f641-4af8-af30-9cb1205ad401 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:17.016811 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1280/2563] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:16 2022] PUT /identity/v3/projects/81cbb2a3fa1541eebd180ba990e69838/users/d5be511ce1994a598756d63284a49846/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:17.021932 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91659fd-420f-4548-96d9-2d2bf7ca7093 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:17.022117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91659fd-420f-4548-96d9-2d2bf7ca7093 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:17.022331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91659fd-420f-4548-96d9-2d2bf7ca7093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:17.042283 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-c91659fd-420f-4548-96d9-2d2bf7ca7093 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:17.043545 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c91659fd-420f-4548-96d9-2d2bf7ca7093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:17.121634 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1284/2564] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:17.129695 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d55e5-f641-4af8-af30-9cb1205ad401 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:17.135729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-238d55e5-f641-4af8-af30-9cb1205ad401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:17.148832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91659fd-420f-4548-96d9-2d2bf7ca7093 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:17.150243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c91659fd-420f-4548-96d9-2d2bf7ca7093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:17.164675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91659fd-420f-4548-96d9-2d2bf7ca7093 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:17.175078 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af0db0-3f5f-4644-b990-9ac492323d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:17.176134 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af0db0-3f5f-4644-b990-9ac492323d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:17.176420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af0db0-3f5f-4644-b990-9ac492323d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:17.176693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af0db0-3f5f-4644-b990-9ac492323d2f admin admin] PATH_INFO: `/v3/users/965cbbdb06c248698ef54527c29767df` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:17.177874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af0db0-3f5f-4644-b990-9ac492323d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=965cbbdb06c248698ef54527c29767df)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:17.184699 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af0db0-3f5f-4644-b990-9ac492323d2f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:17.210637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:17.225379 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d55e5-f641-4af8-af30-9cb1205ad401 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:17.230466 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bc3da-8063-4215-ba7c-4efe0f87f7f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:17.231138 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bc3da-8063-4215-ba7c-4efe0f87f7f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:17.231830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bc3da-8063-4215-ba7c-4efe0f87f7f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:17.232190 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7bc3da-8063-4215-ba7c-4efe0f87f7f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:17.242871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c5af0db0-3f5f-4644-b990-9ac492323d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '965cbbdb06c248698ef54527c29767df'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:17.243097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7bc3da-8063-4215-ba7c-4efe0f87f7f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:17.244538 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7bc3da-8063-4215-ba7c-4efe0f87f7f8 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:17.251711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c5af0db0-3f5f-4644-b990-9ac492323d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '965cbbdb06c248698ef54527c29767df'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:17.252639 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c5af0db0-3f5f-4644-b990-9ac492323d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '965cbbdb06c248698ef54527c29767df'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:17.253279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c5af0db0-3f5f-4644-b990-9ac492323d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '965cbbdb06c248698ef54527c29767df'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:17.261098 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1281/2565] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:17.266582 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-c5af0db0-3f5f-4644-b990-9ac492323d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '965cbbdb06c248698ef54527c29767df'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:17.285735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1285/2566] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:17 2022] DELETE /identity/v3/users/965cbbdb06c248698ef54527c29767df => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:17.556559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bc3da-8063-4215-ba7c-4efe0f87f7f8 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:17.569876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7bc3da-8063-4215-ba7c-4efe0f87f7f8 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:17.652143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924aa939-3cd1-4cb5-9732-52857a0af459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:17.652943 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924aa939-3cd1-4cb5-9732-52857a0af459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:17.653290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924aa939-3cd1-4cb5-9732-52857a0af459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:17.653629 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924aa939-3cd1-4cb5-9732-52857a0af459 admin admin] PATH_INFO: `/v3/projects/c67853e0ad324ac595650e59f9a90f56` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:17.655008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924aa939-3cd1-4cb5-9732-52857a0af459 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c67853e0ad324ac595650e59f9a90f56)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:17.658890 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924aa939-3cd1-4cb5-9732-52857a0af459 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:17.734194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-924aa939-3cd1-4cb5-9732-52857a0af459 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c67853e0ad324ac595650e59f9a90f56', 'request_id': 'req-924aa939-3cd1-4cb5-9732-52857a0af459'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:17.759566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-924aa939-3cd1-4cb5-9732-52857a0af459 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c67853e0ad324ac595650e59f9a90f56', 'request_id': 'req-924aa939-3cd1-4cb5-9732-52857a0af459'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:17.768696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-924aa939-3cd1-4cb5-9732-52857a0af459 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c67853e0ad324ac595650e59f9a90f56', 'request_id': 'req-924aa939-3cd1-4cb5-9732-52857a0af459'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:17.799578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-924aa939-3cd1-4cb5-9732-52857a0af459 admin admin] The token cache is being invalidate because project c67853e0ad324ac595650e59f9a90f56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:17.799801 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-924aa939-3cd1-4cb5-9732-52857a0af459 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:17.800641 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-924aa939-3cd1-4cb5-9732-52857a0af459 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:17.802175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1282/2567] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:17 2022] DELETE /identity/v3/projects/c67853e0ad324ac595650e59f9a90f56 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:17.807375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa9bbe-2643-472f-ba5d-9d5c3c7a2397 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:17.807951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa9bbe-2643-472f-ba5d-9d5c3c7a2397 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:17.807951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa9bbe-2643-472f-ba5d-9d5c3c7a2397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:17.828318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-a1fa9bbe-2643-472f-ba5d-9d5c3c7a2397 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:17.829539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a1fa9bbe-2643-472f-ba5d-9d5c3c7a2397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:17.893658 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1286/2568] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:17.903875 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2935ef0-9d94-45b6-b7b3-2bb25cece2b2 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:17.904425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2935ef0-9d94-45b6-b7b3-2bb25cece2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:17.904738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2935ef0-9d94-45b6-b7b3-2bb25cece2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:17.925528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-e2935ef0-9d94-45b6-b7b3-2bb25cece2b2 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:17.926739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e2935ef0-9d94-45b6-b7b3-2bb25cece2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:17.935288 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1283/2569] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:17.939336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa9bbe-2643-472f-ba5d-9d5c3c7a2397 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:17.949166 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa9bbe-2643-472f-ba5d-9d5c3c7a2397 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:17.954011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4168a7-030b-4e83-b85c-bc7acc61e4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:17.954973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4168a7-030b-4e83-b85c-bc7acc61e4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:17.954973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4168a7-030b-4e83-b85c-bc7acc61e4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:17.955215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4168a7-030b-4e83-b85c-bc7acc61e4c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:17.955877 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4168a7-030b-4e83-b85c-bc7acc61e4c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:15:17.956180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4168a7-030b-4e83-b85c-bc7acc61e4c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:17.957783 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4168a7-030b-4e83-b85c-bc7acc61e4c8 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:17.966724 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:17.970349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1287/2570] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:15:17.974922 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2935ef0-9d94-45b6-b7b3-2bb25cece2b2 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:17.984672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2935ef0-9d94-45b6-b7b3-2bb25cece2b2 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:17.991919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323af025-5b23-43b5-9a89-8ff2fdd8a571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:17.993177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323af025-5b23-43b5-9a89-8ff2fdd8a571 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:17.993625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323af025-5b23-43b5-9a89-8ff2fdd8a571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:17.994286 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323af025-5b23-43b5-9a89-8ff2fdd8a571 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:17.995510 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323af025-5b23-43b5-9a89-8ff2fdd8a571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:17.998038 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323af025-5b23-43b5-9a89-8ff2fdd8a571 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.031881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1284/2571] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:17 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:18.036069 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4168a7-030b-4e83-b85c-bc7acc61e4c8 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.042210 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3271b-e686-4933-8a75-bc35fe4c8812 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.043220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3271b-e686-4933-8a75-bc35fe4c8812 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.043528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3271b-e686-4933-8a75-bc35fe4c8812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.047142 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4168a7-030b-4e83-b85c-bc7acc61e4c8 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.074939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-6ab3271b-e686-4933-8a75-bc35fe4c8812 None None] MFA Rules not processed for user `d5be511ce1994a598756d63284a49846`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:18.076166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6ab3271b-e686-4933-8a75-bc35fe4c8812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:18.117511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191fc4b-acde-4060-a9ab-072f6f7bd620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.118133 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191fc4b-acde-4060-a9ab-072f6f7bd620 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.118268 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:18.118434 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191fc4b-acde-4060-a9ab-072f6f7bd620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.118679 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191fc4b-acde-4060-a9ab-072f6f7bd620 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.119313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191fc4b-acde-4060-a9ab-072f6f7bd620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.121280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191fc4b-acde-4060-a9ab-072f6f7bd620 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.144867 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1288/2572] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:18 2022] POST /identity/v3/users => generated 283 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:18.148632 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191fc4b-acde-4060-a9ab-072f6f7bd620 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.155079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1285/2573] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:18.160004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3271b-e686-4933-8a75-bc35fe4c8812 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.161588 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6ab3271b-e686-4933-8a75-bc35fe4c8812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:18.163060 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191fc4b-acde-4060-a9ab-072f6f7bd620 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.176102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39466b6e-81db-4233-9f57-9832ad79d254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.176491 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3271b-e686-4933-8a75-bc35fe4c8812 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.177030 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39466b6e-81db-4233-9f57-9832ad79d254 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.177312 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39466b6e-81db-4233-9f57-9832ad79d254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.177588 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39466b6e-81db-4233-9f57-9832ad79d254 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.181753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39466b6e-81db-4233-9f57-9832ad79d254 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.181753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39466b6e-81db-4233-9f57-9832ad79d254 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.186267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75fab5-fd79-4832-baf5-886773a876b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.187098 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75fab5-fd79-4832-baf5-886773a876b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.187970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75fab5-fd79-4832-baf5-886773a876b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.187970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75fab5-fd79-4832-baf5-886773a876b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.188640 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75fab5-fd79-4832-baf5-886773a876b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.190785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75fab5-fd79-4832-baf5-886773a876b8 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.192558 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1289/2574] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:18.195920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39466b6e-81db-4233-9f57-9832ad79d254 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.207487 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39466b6e-81db-4233-9f57-9832ad79d254 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.213215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1a4ac-5997-4714-93b2-3db7703e4876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.213790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1a4ac-5997-4714-93b2-3db7703e4876 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.214103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1a4ac-5997-4714-93b2-3db7703e4876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.214384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1a4ac-5997-4714-93b2-3db7703e4876 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.215068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1a4ac-5997-4714-93b2-3db7703e4876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.217137 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1a4ac-5997-4714-93b2-3db7703e4876 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.231482 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1290/2575] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:18.231715 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1286/2576] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:18 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:18.233361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1a4ac-5997-4714-93b2-3db7703e4876 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.235525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75fab5-fd79-4832-baf5-886773a876b8 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.244110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1a4ac-5997-4714-93b2-3db7703e4876 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.300971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75fab5-fd79-4832-baf5-886773a876b8 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.302281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f5bfc-6605-4665-918e-92afaadd0ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.302942 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f5bfc-6605-4665-918e-92afaadd0ca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.303215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f5bfc-6605-4665-918e-92afaadd0ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.303507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f5bfc-6605-4665-918e-92afaadd0ca7 admin admin] PATH_INFO: `/v3/projects/62500495f5784056b051bca753e7c57c/users/c1492ab0f4594f169db6e1b2520f7876/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.305642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcfafd-c2e4-4a5d-8043-88dc4c2a95db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.306261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcfafd-c2e4-4a5d-8043-88dc4c2a95db admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.306558 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcfafd-c2e4-4a5d-8043-88dc4c2a95db admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.306831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bcfafd-c2e4-4a5d-8043-88dc4c2a95db admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.307490 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bcfafd-c2e4-4a5d-8043-88dc4c2a95db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.309458 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bcfafd-c2e4-4a5d-8043-88dc4c2a95db admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.318094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f5bfc-6605-4665-918e-92afaadd0ca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62500495f5784056b051bca753e7c57c, user_id=c1492ab0f4594f169db6e1b2520f7876, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.320845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0f5bfc-6605-4665-918e-92afaadd0ca7 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.332566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1287/2577] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:18 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:18.335035 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1291/2578] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:18 2022] PUT /identity/v3/projects/62500495f5784056b051bca753e7c57c/users/c1492ab0f4594f169db6e1b2520f7876/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:18.336867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcfafd-c2e4-4a5d-8043-88dc4c2a95db admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.345592 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385e872-1460-4321-a7fb-d7a0506f0a4c None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.345874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385e872-1460-4321-a7fb-d7a0506f0a4c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.346146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385e872-1460-4321-a7fb-d7a0506f0a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.347298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bcfafd-c2e4-4a5d-8043-88dc4c2a95db admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.376461 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-0385e872-1460-4321-a7fb-d7a0506f0a4c None None] MFA Rules not processed for user `c1492ab0f4594f169db6e1b2520f7876`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:18.377686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0385e872-1460-4321-a7fb-d7a0506f0a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:18.410097 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00da3d-acc6-4067-8eac-bea6741df60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.410722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00da3d-acc6-4067-8eac-bea6741df60a admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.411000 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00da3d-acc6-4067-8eac-bea6741df60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.411269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00da3d-acc6-4067-8eac-bea6741df60a admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.411954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00da3d-acc6-4067-8eac-bea6741df60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.413602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c00da3d-acc6-4067-8eac-bea6741df60a admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.420003 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:18.423749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1288/2579] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:18.428066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00da3d-acc6-4067-8eac-bea6741df60a admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.439087 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c00da3d-acc6-4067-8eac-bea6741df60a admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.444842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e171b7f-91c5-4b58-8205-7039dab04731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.445484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e171b7f-91c5-4b58-8205-7039dab04731 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.445757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e171b7f-91c5-4b58-8205-7039dab04731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.446016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e171b7f-91c5-4b58-8205-7039dab04731 admin admin] PATH_INFO: `/v3/projects/f904fa9074924fadab446d6925f23596/users/6650001c51db419dbfd9b3c7b1dd2467/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.446928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1292/2580] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:18.451606 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385e872-1460-4321-a7fb-d7a0506f0a4c None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.461929 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385e872-1460-4321-a7fb-d7a0506f0a4c None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.462172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e171b7f-91c5-4b58-8205-7039dab04731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f904fa9074924fadab446d6925f23596, user_id=6650001c51db419dbfd9b3c7b1dd2467, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.464878 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e171b7f-91c5-4b58-8205-7039dab04731 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.466651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222583fe-e240-4185-ab3d-6e8f9ad29ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.467613 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222583fe-e240-4185-ab3d-6e8f9ad29ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.468265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222583fe-e240-4185-ab3d-6e8f9ad29ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.468537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222583fe-e240-4185-ab3d-6e8f9ad29ef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.469281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222583fe-e240-4185-ab3d-6e8f9ad29ef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.471534 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222583fe-e240-4185-ab3d-6e8f9ad29ef4 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.479188 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1289/2581] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:18 2022] PUT /identity/v3/projects/f904fa9074924fadab446d6925f23596/users/6650001c51db419dbfd9b3c7b1dd2467/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:18.483385 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e171b7f-91c5-4b58-8205-7039dab04731 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.500450 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1293/2582] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:18 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:18.503306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:18.504727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222583fe-e240-4185-ab3d-6e8f9ad29ef4 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.548647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e171b7f-91c5-4b58-8205-7039dab04731 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.567731 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222583fe-e240-4185-ab3d-6e8f9ad29ef4 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.572582 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00020117-1060-4076-a509-78a517c47857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.573189 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00020117-1060-4076-a509-78a517c47857 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.573465 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00020117-1060-4076-a509-78a517c47857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.573735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00020117-1060-4076-a509-78a517c47857 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.574399 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00020117-1060-4076-a509-78a517c47857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.576378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00020117-1060-4076-a509-78a517c47857 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.598129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1294/2583] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:18 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:18.602048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00020117-1060-4076-a509-78a517c47857 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.607950 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260210ca-69f2-42a5-b4c9-d409e0f56036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.608561 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260210ca-69f2-42a5-b4c9-d409e0f56036 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.608833 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260210ca-69f2-42a5-b4c9-d409e0f56036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.609096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260210ca-69f2-42a5-b4c9-d409e0f56036 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.609737 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260210ca-69f2-42a5-b4c9-d409e0f56036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.611353 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260210ca-69f2-42a5-b4c9-d409e0f56036 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.611506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00020117-1060-4076-a509-78a517c47857 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.616122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3a546-a288-4a12-9de3-f874e42a4b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.616739 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3a546-a288-4a12-9de3-f874e42a4b07 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.617015 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3a546-a288-4a12-9de3-f874e42a4b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.617283 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3a546-a288-4a12-9de3-f874e42a4b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.617972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3a546-a288-4a12-9de3-f874e42a4b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.619573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a3a546-a288-4a12-9de3-f874e42a4b07 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.622191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1290/2584] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:18.626082 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260210ca-69f2-42a5-b4c9-d409e0f56036 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.633067 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1295/2585] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:18.634077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3a546-a288-4a12-9de3-f874e42a4b07 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.636137 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260210ca-69f2-42a5-b4c9-d409e0f56036 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.641166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e4274-08c7-49c3-b6e7-5d9a4fa43e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.641815 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e4274-08c7-49c3-b6e7-5d9a4fa43e9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.642091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e4274-08c7-49c3-b6e7-5d9a4fa43e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.642362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e4274-08c7-49c3-b6e7-5d9a4fa43e9a admin admin] PATH_INFO: `/v3/projects/f904fa9074924fadab446d6925f23596/users/6650001c51db419dbfd9b3c7b1dd2467/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.643637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a3a546-a288-4a12-9de3-f874e42a4b07 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.644470 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449e4274-08c7-49c3-b6e7-5d9a4fa43e9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f904fa9074924fadab446d6925f23596, user_id=6650001c51db419dbfd9b3c7b1dd2467, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.647108 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449e4274-08c7-49c3-b6e7-5d9a4fa43e9a admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.648413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fad419-4b6a-441b-9e8f-eaea4d6be47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.649045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fad419-4b6a-441b-9e8f-eaea4d6be47e admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.649288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fad419-4b6a-441b-9e8f-eaea4d6be47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.649592 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fad419-4b6a-441b-9e8f-eaea4d6be47e admin admin] PATH_INFO: `/v3/projects/2720d4c7a6cd458aa3b39c3ea5e03492/users/5c9b805eff7e4a8dbc3c205f988f785a/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.657742 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1291/2586] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:18 2022] PUT /identity/v3/projects/f904fa9074924fadab446d6925f23596/users/6650001c51db419dbfd9b3c7b1dd2467/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:18.664336 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fad419-4b6a-441b-9e8f-eaea4d6be47e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2720d4c7a6cd458aa3b39c3ea5e03492, user_id=5c9b805eff7e4a8dbc3c205f988f785a, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.667079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fad419-4b6a-441b-9e8f-eaea4d6be47e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.682124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1296/2587] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:18 2022] PUT /identity/v3/projects/2720d4c7a6cd458aa3b39c3ea5e03492/users/5c9b805eff7e4a8dbc3c205f988f785a/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:18.686281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e4274-08c7-49c3-b6e7-5d9a4fa43e9a admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.753648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449e4274-08c7-49c3-b6e7-5d9a4fa43e9a admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.806568 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:18.816645 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81367862-df59-40b4-9900-2d9331268d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.817306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81367862-df59-40b4-9900-2d9331268d0c admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.817589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81367862-df59-40b4-9900-2d9331268d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.817888 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81367862-df59-40b4-9900-2d9331268d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.818685 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81367862-df59-40b4-9900-2d9331268d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.820406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81367862-df59-40b4-9900-2d9331268d0c admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.834211 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1292/2588] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:18.835504 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fad419-4b6a-441b-9e8f-eaea4d6be47e admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:18.846564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fad419-4b6a-441b-9e8f-eaea4d6be47e admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:18.851409 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741f674-dc89-48cc-849d-b83f6631cfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:18.852050 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741f674-dc89-48cc-849d-b83f6631cfb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.852328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741f674-dc89-48cc-849d-b83f6631cfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.852599 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741f674-dc89-48cc-849d-b83f6631cfb8 admin admin] PATH_INFO: `/v3/projects/2720d4c7a6cd458aa3b39c3ea5e03492/users/5c9b805eff7e4a8dbc3c205f988f785a/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.854515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741f674-dc89-48cc-849d-b83f6631cfb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2720d4c7a6cd458aa3b39c3ea5e03492, user_id=5c9b805eff7e4a8dbc3c205f988f785a, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:18.857180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741f674-dc89-48cc-849d-b83f6631cfb8 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:18.868654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1297/2589] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:18 2022] PUT /identity/v3/projects/2720d4c7a6cd458aa3b39c3ea5e03492/users/5c9b805eff7e4a8dbc3c205f988f785a/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:18.880830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754bda5-2e06-477e-8dc4-e3aa763a177b None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:18.881193 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754bda5-2e06-477e-8dc4-e3aa763a177b None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:18.881471 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754bda5-2e06-477e-8dc4-e3aa763a177b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:18.913403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-1754bda5-2e06-477e-8dc4-e3aa763a177b None None] MFA Rules not processed for user `5c9b805eff7e4a8dbc3c205f988f785a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:18.914750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-1754bda5-2e06-477e-8dc4-e3aa763a177b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:18.992845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1293/2590] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:19.005548 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741f674-dc89-48cc-849d-b83f6631cfb8 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:19.007758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a741f674-dc89-48cc-849d-b83f6631cfb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:19.112400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741f674-dc89-48cc-849d-b83f6631cfb8 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:19.116346 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb678116-1f07-4df8-8ec6-8be6b4ac537b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:19.117086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb678116-1f07-4df8-8ec6-8be6b4ac537b service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:19.117441 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb678116-1f07-4df8-8ec6-8be6b4ac537b service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:19.117911 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb678116-1f07-4df8-8ec6-8be6b4ac537b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:19.129254 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb678116-1f07-4df8-8ec6-8be6b4ac537b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:19.130719 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb678116-1f07-4df8-8ec6-8be6b4ac537b service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:19.147966 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1298/2591] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:19.230758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754bda5-2e06-477e-8dc4-e3aa763a177b None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:19.240709 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754bda5-2e06-477e-8dc4-e3aa763a177b None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:19.244486 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89385d59-e0d4-4621-a0d2-0741a12bda91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:19.245227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89385d59-e0d4-4621-a0d2-0741a12bda91 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:19.245610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89385d59-e0d4-4621-a0d2-0741a12bda91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:19.245967 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89385d59-e0d4-4621-a0d2-0741a12bda91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:19.325027 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89385d59-e0d4-4621-a0d2-0741a12bda91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:19.327287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89385d59-e0d4-4621-a0d2-0741a12bda91 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:19.339514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1294/2592] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:19.353995 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a0ae8-b9d2-4d69-b443-296a4b029a67 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:19.354281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a0ae8-b9d2-4d69-b443-296a4b029a67 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:19.354662 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a0ae8-b9d2-4d69-b443-296a4b029a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:19.362765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:19.398725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-a54a0ae8-b9d2-4d69-b443-296a4b029a67 None None] MFA Rules not processed for user `6650001c51db419dbfd9b3c7b1dd2467`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:19.400092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a54a0ae8-b9d2-4d69-b443-296a4b029a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:19.491482 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1299/2593] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:19.506225 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89385d59-e0d4-4621-a0d2-0741a12bda91 service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:19.516434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89385d59-e0d4-4621-a0d2-0741a12bda91 service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:19.529779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934abf7-1dd2-4c42-aadf-05ce6954c4ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:19.530427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934abf7-1dd2-4c42-aadf-05ce6954c4ed service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:19.530708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934abf7-1dd2-4c42-aadf-05ce6954c4ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:19.530983 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e934abf7-1dd2-4c42-aadf-05ce6954c4ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:19.619117 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e934abf7-1dd2-4c42-aadf-05ce6954c4ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:19.620672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e934abf7-1dd2-4c42-aadf-05ce6954c4ed service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:19.631203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1295/2594] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:21.027141 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a0ae8-b9d2-4d69-b443-296a4b029a67 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:21.038346 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a0ae8-b9d2-4d69-b443-296a4b029a67 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:21.042371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49518b25-b86d-4dca-b31c-51dca223c77a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:21.043061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49518b25-b86d-4dca-b31c-51dca223c77a service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:21.043429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49518b25-b86d-4dca-b31c-51dca223c77a service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:21.043779 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49518b25-b86d-4dca-b31c-51dca223c77a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:21.053685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49518b25-b86d-4dca-b31c-51dca223c77a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:21.055156 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49518b25-b86d-4dca-b31c-51dca223c77a service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:21.066808 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1300/2595] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:26.887183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934abf7-1dd2-4c42-aadf-05ce6954c4ed service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:26.888851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e934abf7-1dd2-4c42-aadf-05ce6954c4ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:26.904263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:26.949403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e934abf7-1dd2-4c42-aadf-05ce6954c4ed service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:26.953106 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910781f0-30cb-49de-a644-4a00708c1ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:26.953710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910781f0-30cb-49de-a644-4a00708c1ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:26.953979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910781f0-30cb-49de-a644-4a00708c1ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:26.954243 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910781f0-30cb-49de-a644-4a00708c1ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:26.955986 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-910781f0-30cb-49de-a644-4a00708c1ec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:27.021707 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910781f0-30cb-49de-a644-4a00708c1ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:27.023528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910781f0-30cb-49de-a644-4a00708c1ec3 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:27.033370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1296/2596] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:29.908533 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49518b25-b86d-4dca-b31c-51dca223c77a service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:29.921636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49518b25-b86d-4dca-b31c-51dca223c77a service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:29.935143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670c5eb-1e06-4049-94ba-24f060e0b1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:29.935942 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670c5eb-1e06-4049-94ba-24f060e0b1a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:29.936285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670c5eb-1e06-4049-94ba-24f060e0b1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:29.936614 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670c5eb-1e06-4049-94ba-24f060e0b1a3 admin admin] PATH_INFO: `/v3/users/c1492ab0f4594f169db6e1b2520f7876` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:29.937899 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670c5eb-1e06-4049-94ba-24f060e0b1a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1492ab0f4594f169db6e1b2520f7876)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:29.940252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670c5eb-1e06-4049-94ba-24f060e0b1a3 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:30.003808 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-9670c5eb-1e06-4049-94ba-24f060e0b1a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1492ab0f4594f169db6e1b2520f7876'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.015430 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-9670c5eb-1e06-4049-94ba-24f060e0b1a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1492ab0f4594f169db6e1b2520f7876'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.016309 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-9670c5eb-1e06-4049-94ba-24f060e0b1a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1492ab0f4594f169db6e1b2520f7876'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.017206 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-9670c5eb-1e06-4049-94ba-24f060e0b1a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1492ab0f4594f169db6e1b2520f7876'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.028531 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-9670c5eb-1e06-4049-94ba-24f060e0b1a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1492ab0f4594f169db6e1b2520f7876'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.050484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1301/2597] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:29 2022] DELETE /identity/v3/users/c1492ab0f4594f169db6e1b2520f7876 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:30.055177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30682dc2-9b47-4fe0-9e2d-3811f22e21f9 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:30.056396 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30682dc2-9b47-4fe0-9e2d-3811f22e21f9 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:30.056822 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30682dc2-9b47-4fe0-9e2d-3811f22e21f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:30.082741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-30682dc2-9b47-4fe0-9e2d-3811f22e21f9 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:30.084309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-30682dc2-9b47-4fe0-9e2d-3811f22e21f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:30.211429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1297/2598] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:30.222249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670c5eb-1e06-4049-94ba-24f060e0b1a3 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:30.224414 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9670c5eb-1e06-4049-94ba-24f060e0b1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:30.323782 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:30.336539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670c5eb-1e06-4049-94ba-24f060e0b1a3 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:30.346411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a37e3-4653-41d8-a6cb-527906accce6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:30.347758 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a37e3-4653-41d8-a6cb-527906accce6 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:30.348256 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a37e3-4653-41d8-a6cb-527906accce6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:30.349455 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a37e3-4653-41d8-a6cb-527906accce6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:30.362709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a37e3-4653-41d8-a6cb-527906accce6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:30.364512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a37e3-4653-41d8-a6cb-527906accce6 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:30.378021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1302/2599] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:30.479745 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30682dc2-9b47-4fe0-9e2d-3811f22e21f9 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:30.481311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-30682dc2-9b47-4fe0-9e2d-3811f22e21f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:30.499234 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30682dc2-9b47-4fe0-9e2d-3811f22e21f9 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:30.505455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb808058-4181-4ecc-9943-460779224ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:30.506207 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb808058-4181-4ecc-9943-460779224ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:30.506500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb808058-4181-4ecc-9943-460779224ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:30.506826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb808058-4181-4ecc-9943-460779224ff1 admin admin] PATH_INFO: `/v3/users/c89859a30a2442b8ae13fe0444782d9c` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:30.508024 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb808058-4181-4ecc-9943-460779224ff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c89859a30a2442b8ae13fe0444782d9c)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:30.510925 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb808058-4181-4ecc-9943-460779224ff1 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:30.572404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cb808058-4181-4ecc-9943-460779224ff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c89859a30a2442b8ae13fe0444782d9c'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.583268 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cb808058-4181-4ecc-9943-460779224ff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c89859a30a2442b8ae13fe0444782d9c'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.583920 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cb808058-4181-4ecc-9943-460779224ff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c89859a30a2442b8ae13fe0444782d9c'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.584516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cb808058-4181-4ecc-9943-460779224ff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c89859a30a2442b8ae13fe0444782d9c'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.589719 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:30.601225 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cb808058-4181-4ecc-9943-460779224ff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c89859a30a2442b8ae13fe0444782d9c'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.617391 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1298/2600] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:30 2022] DELETE /identity/v3/users/c89859a30a2442b8ae13fe0444782d9c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:30.698031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a37e3-4653-41d8-a6cb-527906accce6 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:30.699866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-016a37e3-4653-41d8-a6cb-527906accce6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:30.714427 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a37e3-4653-41d8-a6cb-527906accce6 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:30.775281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4243e5c-d9bd-4293-a2c2-595bd69e429f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:30.775884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4243e5c-d9bd-4293-a2c2-595bd69e429f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:30.776163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4243e5c-d9bd-4293-a2c2-595bd69e429f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:30.776434 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4243e5c-d9bd-4293-a2c2-595bd69e429f admin admin] PATH_INFO: `/v3/projects/62500495f5784056b051bca753e7c57c` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:30.777577 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4243e5c-d9bd-4293-a2c2-595bd69e429f admin admin] RBAC: Authorizing `identity:delete_project(project_id=62500495f5784056b051bca753e7c57c)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:30.779790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4243e5c-d9bd-4293-a2c2-595bd69e429f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:30.810195 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb808058-4181-4ecc-9943-460779224ff1 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:30.811874 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-cb808058-4181-4ecc-9943-460779224ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:30.826072 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb808058-4181-4ecc-9943-460779224ff1 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:30.835101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a4243e5c-d9bd-4293-a2c2-595bd69e429f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62500495f5784056b051bca753e7c57c', 'request_id': 'req-a4243e5c-d9bd-4293-a2c2-595bd69e429f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.845067 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a4243e5c-d9bd-4293-a2c2-595bd69e429f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62500495f5784056b051bca753e7c57c', 'request_id': 'req-a4243e5c-d9bd-4293-a2c2-595bd69e429f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.850768 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a4243e5c-d9bd-4293-a2c2-595bd69e429f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62500495f5784056b051bca753e7c57c', 'request_id': 'req-a4243e5c-d9bd-4293-a2c2-595bd69e429f'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.862083 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a4243e5c-d9bd-4293-a2c2-595bd69e429f admin admin] The token cache is being invalidate because project 62500495f5784056b051bca753e7c57c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:30.862404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a4243e5c-d9bd-4293-a2c2-595bd69e429f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.863055 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-a4243e5c-d9bd-4293-a2c2-595bd69e429f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.864661 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1303/2601] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:30 2022] DELETE /identity/v3/projects/62500495f5784056b051bca753e7c57c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:30.868565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4243e5c-d9bd-4293-a2c2-595bd69e429f admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:30.870265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-a4243e5c-d9bd-4293-a2c2-595bd69e429f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:30.885931 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1135321-07af-4384-abe4-28f8b5218be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:30.886578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1135321-07af-4384-abe4-28f8b5218be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:30.886858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1135321-07af-4384-abe4-28f8b5218be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:30.887132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1135321-07af-4384-abe4-28f8b5218be4 admin admin] PATH_INFO: `/v3/projects/52651960a8b6494193c5e8abc658a2dd` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:30.888297 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1135321-07af-4384-abe4-28f8b5218be4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52651960a8b6494193c5e8abc658a2dd)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:30.890320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1135321-07af-4384-abe4-28f8b5218be4 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:30.937355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4243e5c-d9bd-4293-a2c2-595bd69e429f admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:30.939783 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e1135321-07af-4384-abe4-28f8b5218be4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52651960a8b6494193c5e8abc658a2dd', 'request_id': 'req-e1135321-07af-4384-abe4-28f8b5218be4'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.948004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e1135321-07af-4384-abe4-28f8b5218be4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52651960a8b6494193c5e8abc658a2dd', 'request_id': 'req-e1135321-07af-4384-abe4-28f8b5218be4'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.954087 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e1135321-07af-4384-abe4-28f8b5218be4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52651960a8b6494193c5e8abc658a2dd', 'request_id': 'req-e1135321-07af-4384-abe4-28f8b5218be4'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.963280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:30.965654 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e1135321-07af-4384-abe4-28f8b5218be4 admin admin] The token cache is being invalidate because project 52651960a8b6494193c5e8abc658a2dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:30.965940 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e1135321-07af-4384-abe4-28f8b5218be4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.966591 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e1135321-07af-4384-abe4-28f8b5218be4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:30.968211 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1299/2602] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:30 2022] DELETE /identity/v3/projects/52651960a8b6494193c5e8abc658a2dd => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:30.973085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd4b78-563d-4f7f-a95d-c21200c6637e None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:30.973368 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd4b78-563d-4f7f-a95d-c21200c6637e None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:30.973643 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd4b78-563d-4f7f-a95d-c21200c6637e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:30.992918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-45fd4b78-563d-4f7f-a95d-c21200c6637e None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:30.994130 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-45fd4b78-563d-4f7f-a95d-c21200c6637e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:30.998566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684768de-6582-4ab7-8ff9-98380fc0e763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:30.999245 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684768de-6582-4ab7-8ff9-98380fc0e763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:30.999633 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684768de-6582-4ab7-8ff9-98380fc0e763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:30.999908 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684768de-6582-4ab7-8ff9-98380fc0e763 admin admin] PATH_INFO: `/v3/users/5c9b805eff7e4a8dbc3c205f988f785a` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.001054 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684768de-6582-4ab7-8ff9-98380fc0e763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c9b805eff7e4a8dbc3c205f988f785a)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:31.002061 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1300/2603] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:31.003022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684768de-6582-4ab7-8ff9-98380fc0e763 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:31.009932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6ada8-c992-4225-b254-a4d2501a2dff None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.010212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6ada8-c992-4225-b254-a4d2501a2dff None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.010483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6ada8-c992-4225-b254-a4d2501a2dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.032062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-25c6ada8-c992-4225-b254-a4d2501a2dff None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:31.033464 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-25c6ada8-c992-4225-b254-a4d2501a2dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:31.042003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1301/2604] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:31.046282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6ada8-c992-4225-b254-a4d2501a2dff None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:31.050175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-684768de-6582-4ab7-8ff9-98380fc0e763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c9b805eff7e4a8dbc3c205f988f785a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:31.055550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6ada8-c992-4225-b254-a4d2501a2dff None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:31.058960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-684768de-6582-4ab7-8ff9-98380fc0e763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c9b805eff7e4a8dbc3c205f988f785a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:31.060146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-684768de-6582-4ab7-8ff9-98380fc0e763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c9b805eff7e4a8dbc3c205f988f785a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:31.060664 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f821a-7667-4dce-b358-3ec35b4b8847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:31.060935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-684768de-6582-4ab7-8ff9-98380fc0e763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c9b805eff7e4a8dbc3c205f988f785a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:31.061266 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f821a-7667-4dce-b358-3ec35b4b8847 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.061534 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f821a-7667-4dce-b358-3ec35b4b8847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.061801 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4f821a-7667-4dce-b358-3ec35b4b8847 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.062444 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f821a-7667-4dce-b358-3ec35b4b8847 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:15:31.062738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f821a-7667-4dce-b358-3ec35b4b8847 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:31.064475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4f821a-7667-4dce-b358-3ec35b4b8847 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:31.069069 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-684768de-6582-4ab7-8ff9-98380fc0e763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c9b805eff7e4a8dbc3c205f988f785a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:31.076769 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1302/2605] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:15:31.081115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f821a-7667-4dce-b358-3ec35b4b8847 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:31.083002 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-af4f821a-7667-4dce-b358-3ec35b4b8847 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:31.087243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1304/2606] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:30 2022] DELETE /identity/v3/users/5c9b805eff7e4a8dbc3c205f988f785a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:31.095911 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4f821a-7667-4dce-b358-3ec35b4b8847 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:31.134592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:31.154850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e83ec8-4098-47fa-9d7d-aff5b4197bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:31.155525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e83ec8-4098-47fa-9d7d-aff5b4197bce admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.155829 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e83ec8-4098-47fa-9d7d-aff5b4197bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.156095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e83ec8-4098-47fa-9d7d-aff5b4197bce admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.156735 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e83ec8-4098-47fa-9d7d-aff5b4197bce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:31.158653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e83ec8-4098-47fa-9d7d-aff5b4197bce admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:31.188537 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1303/2607] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:31 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:31.192728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684768de-6582-4ab7-8ff9-98380fc0e763 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:31.203280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684768de-6582-4ab7-8ff9-98380fc0e763 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:31.263317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bec7d9-eb96-4939-8510-cb119d4fe77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:31.263872 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bec7d9-eb96-4939-8510-cb119d4fe77e admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.264162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bec7d9-eb96-4939-8510-cb119d4fe77e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.264434 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bec7d9-eb96-4939-8510-cb119d4fe77e admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.265085 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bec7d9-eb96-4939-8510-cb119d4fe77e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:31.267002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bec7d9-eb96-4939-8510-cb119d4fe77e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:31.294030 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1305/2608] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:31 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:31.298123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e83ec8-4098-47fa-9d7d-aff5b4197bce admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:31.308657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e83ec8-4098-47fa-9d7d-aff5b4197bce admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:31.314364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41206da-5517-49d7-a4b8-c1055602f34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:31.315402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41206da-5517-49d7-a4b8-c1055602f34b admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.315713 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41206da-5517-49d7-a4b8-c1055602f34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.315985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41206da-5517-49d7-a4b8-c1055602f34b admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.316636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41206da-5517-49d7-a4b8-c1055602f34b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:31.318263 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41206da-5517-49d7-a4b8-c1055602f34b admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:31.329390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1304/2609] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:31.333354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bec7d9-eb96-4939-8510-cb119d4fe77e admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:31.365443 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bec7d9-eb96-4939-8510-cb119d4fe77e admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:31.370951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41206da-5517-49d7-a4b8-c1055602f34b admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:31.372600 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c2a71-ec93-49d2-93c4-83a2e7edc5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:31.372782 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d41206da-5517-49d7-a4b8-c1055602f34b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:31.373490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c2a71-ec93-49d2-93c4-83a2e7edc5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.373858 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c2a71-ec93-49d2-93c4-83a2e7edc5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.374295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c2a71-ec93-49d2-93c4-83a2e7edc5f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.375178 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c2a71-ec93-49d2-93c4-83a2e7edc5f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:31.377180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c2a71-ec93-49d2-93c4-83a2e7edc5f3 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:31.407664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1306/2610] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:31.410654 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41206da-5517-49d7-a4b8-c1055602f34b admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:31.412994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c2a71-ec93-49d2-93c4-83a2e7edc5f3 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:31.418131 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd04695-7271-4daf-ba37-7e16f7a2a06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:31.419032 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd04695-7271-4daf-ba37-7e16f7a2a06a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.419540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd04695-7271-4daf-ba37-7e16f7a2a06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.420059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd04695-7271-4daf-ba37-7e16f7a2a06a admin admin] PATH_INFO: `/v3/projects/2720d4c7a6cd458aa3b39c3ea5e03492` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.422284 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd04695-7271-4daf-ba37-7e16f7a2a06a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2720d4c7a6cd458aa3b39c3ea5e03492)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:31.424151 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd04695-7271-4daf-ba37-7e16f7a2a06a admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:31.425159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c2a71-ec93-49d2-93c4-83a2e7edc5f3 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:31.434664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b65418-ef7f-4305-99f7-ea39312c16d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:31.435526 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b65418-ef7f-4305-99f7-ea39312c16d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.435760 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b65418-ef7f-4305-99f7-ea39312c16d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.436447 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b65418-ef7f-4305-99f7-ea39312c16d7 admin admin] PATH_INFO: `/v3/projects/f079600e361845c6aac08d07151d1dea/users/6d2416bd945d4acd9bb263cab2223f3e/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.455734 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b65418-ef7f-4305-99f7-ea39312c16d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f079600e361845c6aac08d07151d1dea, user_id=6d2416bd945d4acd9bb263cab2223f3e, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:31.458961 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b65418-ef7f-4305-99f7-ea39312c16d7 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:31.472136 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:31.484037 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1307/2611] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:31 2022] PUT /identity/v3/projects/f079600e361845c6aac08d07151d1dea/users/6d2416bd945d4acd9bb263cab2223f3e/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:31.498594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127ebbc-ea05-4dfe-b470-6c4b0d1d842b None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.499330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127ebbc-ea05-4dfe-b470-6c4b0d1d842b None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.505411 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cfd04695-7271-4daf-ba37-7e16f7a2a06a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2720d4c7a6cd458aa3b39c3ea5e03492', 'request_id': 'req-cfd04695-7271-4daf-ba37-7e16f7a2a06a'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:31.512893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127ebbc-ea05-4dfe-b470-6c4b0d1d842b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.516641 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cfd04695-7271-4daf-ba37-7e16f7a2a06a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2720d4c7a6cd458aa3b39c3ea5e03492', 'request_id': 'req-cfd04695-7271-4daf-ba37-7e16f7a2a06a'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:31.526204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cfd04695-7271-4daf-ba37-7e16f7a2a06a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2720d4c7a6cd458aa3b39c3ea5e03492', 'request_id': 'req-cfd04695-7271-4daf-ba37-7e16f7a2a06a'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:31.539951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cfd04695-7271-4daf-ba37-7e16f7a2a06a admin admin] The token cache is being invalidate because project 2720d4c7a6cd458aa3b39c3ea5e03492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:31.540255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cfd04695-7271-4daf-ba37-7e16f7a2a06a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:31.541004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-cfd04695-7271-4daf-ba37-7e16f7a2a06a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:31.542608 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1305/2612] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:31 2022] DELETE /identity/v3/projects/2720d4c7a6cd458aa3b39c3ea5e03492 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:31.548041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-e127ebbc-ea05-4dfe-b470-6c4b0d1d842b None None] MFA Rules not processed for user `6d2416bd945d4acd9bb263cab2223f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:31.549147 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f303b1b-afd6-4133-b1d9-c3cd98dbd5cd None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.549465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f303b1b-afd6-4133-b1d9-c3cd98dbd5cd None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.549585 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e127ebbc-ea05-4dfe-b470-6c4b0d1d842b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:31.549737 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f303b1b-afd6-4133-b1d9-c3cd98dbd5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.573112 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-0f303b1b-afd6-4133-b1d9-c3cd98dbd5cd None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:31.575285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0f303b1b-afd6-4133-b1d9-c3cd98dbd5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:31.629111 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1308/2613] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:31.640573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127ebbc-ea05-4dfe-b470-6c4b0d1d842b None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:31.641919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e127ebbc-ea05-4dfe-b470-6c4b0d1d842b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:31.645978 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1306/2614] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:31.653785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93eb410-0cb0-4bc6-bda4-86033ad0cf5a None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.654110 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93eb410-0cb0-4bc6-bda4-86033ad0cf5a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.654399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93eb410-0cb0-4bc6-bda4-86033ad0cf5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.673412 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-e93eb410-0cb0-4bc6-bda4-86033ad0cf5a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:31.674602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e93eb410-0cb0-4bc6-bda4-86033ad0cf5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:31.682727 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1307/2615] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:31.687525 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93eb410-0cb0-4bc6-bda4-86033ad0cf5a None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:31.697096 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93eb410-0cb0-4bc6-bda4-86033ad0cf5a None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:31.702799 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543c333-d289-4858-abc5-25dcdf234e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:31.703529 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543c333-d289-4858-abc5-25dcdf234e51 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.703819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543c333-d289-4858-abc5-25dcdf234e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.704125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543c333-d289-4858-abc5-25dcdf234e51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.705103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543c333-d289-4858-abc5-25dcdf234e51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:15:31.705387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543c333-d289-4858-abc5-25dcdf234e51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:31.707105 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543c333-d289-4858-abc5-25dcdf234e51 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:31.718683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127ebbc-ea05-4dfe-b470-6c4b0d1d842b None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:31.721580 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1308/2616] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:15:31.726115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543c333-d289-4858-abc5-25dcdf234e51 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:31.728404 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91526b-0074-484d-8a23-c926cf535fe7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:31.729139 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91526b-0074-484d-8a23-c926cf535fe7 service nova] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.729457 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91526b-0074-484d-8a23-c926cf535fe7 service nova] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.729813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd91526b-0074-484d-8a23-c926cf535fe7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.735935 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543c333-d289-4858-abc5-25dcdf234e51 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:31.738467 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91526b-0074-484d-8a23-c926cf535fe7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:31.740131 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd91526b-0074-484d-8a23-c926cf535fe7 service nova] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:31.740814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3748f-047a-4a3c-8051-8ff9d8e1591d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:31.741410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3748f-047a-4a3c-8051-8ff9d8e1591d admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.741690 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3748f-047a-4a3c-8051-8ff9d8e1591d admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.741990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f3748f-047a-4a3c-8051-8ff9d8e1591d admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.742637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3748f-047a-4a3c-8051-8ff9d8e1591d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:31.744639 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f3748f-047a-4a3c-8051-8ff9d8e1591d admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:31.749445 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1309/2617] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:31.763414 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:31.780124 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1309/2618] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:31 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:31.782013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91526b-0074-484d-8a23-c926cf535fe7 service nova] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:31.794250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd91526b-0074-484d-8a23-c926cf535fe7 service nova] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:31.851010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337ece5-0144-4815-a6c1-4fbd3657d0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:31.851712 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337ece5-0144-4815-a6c1-4fbd3657d0da admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.852026 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337ece5-0144-4815-a6c1-4fbd3657d0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.852265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337ece5-0144-4815-a6c1-4fbd3657d0da admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.852938 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337ece5-0144-4815-a6c1-4fbd3657d0da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:31.854993 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337ece5-0144-4815-a6c1-4fbd3657d0da admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:31.880735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1310/2619] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:31 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:31.882346 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3748f-047a-4a3c-8051-8ff9d8e1591d admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:31.893633 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f3748f-047a-4a3c-8051-8ff9d8e1591d admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:31.899209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ae1f1-57f8-45b2-bd5d-f94e5aad04a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:31.899863 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ae1f1-57f8-45b2-bd5d-f94e5aad04a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.900182 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ae1f1-57f8-45b2-bd5d-f94e5aad04a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.900566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ae1f1-57f8-45b2-bd5d-f94e5aad04a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.901224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ae1f1-57f8-45b2-bd5d-f94e5aad04a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:31.902871 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ae1f1-57f8-45b2-bd5d-f94e5aad04a5 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:31.914029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1310/2620] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:31.918619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337ece5-0144-4815-a6c1-4fbd3657d0da admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:31.925777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:31.930075 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337ece5-0144-4815-a6c1-4fbd3657d0da admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:31.936464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d023b62-b6ea-49f0-81de-9a9b6a13965a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:31.937102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d023b62-b6ea-49f0-81de-9a9b6a13965a admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.937380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d023b62-b6ea-49f0-81de-9a9b6a13965a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.937946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d023b62-b6ea-49f0-81de-9a9b6a13965a admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.938318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d023b62-b6ea-49f0-81de-9a9b6a13965a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:31.939953 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d023b62-b6ea-49f0-81de-9a9b6a13965a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:31.950365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1311/2621] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:31.955041 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ae1f1-57f8-45b2-bd5d-f94e5aad04a5 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:31.965506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ae1f1-57f8-45b2-bd5d-f94e5aad04a5 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:31.970224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27fd0e-e63d-4482-85d5-62c3d2e94fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:31.970880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27fd0e-e63d-4482-85d5-62c3d2e94fdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:31.971160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27fd0e-e63d-4482-85d5-62c3d2e94fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:31.971435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a27fd0e-e63d-4482-85d5-62c3d2e94fdb admin admin] PATH_INFO: `/v3/projects/03fd5530dc0e40d386aebc3a185ec633/users/a5b07dab746341caadb5624137190975/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:31.988433 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27fd0e-e63d-4482-85d5-62c3d2e94fdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=03fd5530dc0e40d386aebc3a185ec633, user_id=a5b07dab746341caadb5624137190975, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:31.991085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a27fd0e-e63d-4482-85d5-62c3d2e94fdb admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:32.006398 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1311/2622] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:31 2022] PUT /identity/v3/projects/03fd5530dc0e40d386aebc3a185ec633/users/a5b07dab746341caadb5624137190975/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:32.017220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03fa63-4539-4429-bdf7-b0cc1671ac30 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:32.017553 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03fa63-4539-4429-bdf7-b0cc1671ac30 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:32.017779 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03fa63-4539-4429-bdf7-b0cc1671ac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:32.048203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-0c03fa63-4539-4429-bdf7-b0cc1671ac30 None None] MFA Rules not processed for user `a5b07dab746341caadb5624137190975`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:32.049564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0c03fa63-4539-4429-bdf7-b0cc1671ac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:32.125744 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1312/2623] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:32.130488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27fd0e-e63d-4482-85d5-62c3d2e94fdb admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:32.194711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a27fd0e-e63d-4482-85d5-62c3d2e94fdb admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:32.200220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d823f6-dbc5-48cc-857c-247306762c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:32.200919 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d823f6-dbc5-48cc-857c-247306762c7e admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:32.201309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d823f6-dbc5-48cc-857c-247306762c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:32.201585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d823f6-dbc5-48cc-857c-247306762c7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:32.202223 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d823f6-dbc5-48cc-857c-247306762c7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:32.204190 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d823f6-dbc5-48cc-857c-247306762c7e admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:32.231922 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1312/2624] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:32 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:32.236243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03fa63-4539-4429-bdf7-b0cc1671ac30 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:32.247804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03fa63-4539-4429-bdf7-b0cc1671ac30 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:32.306124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe82b23-680e-4c2b-bbc9-f501ed31f756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:32.306735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe82b23-680e-4c2b-bbc9-f501ed31f756 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:32.307006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe82b23-680e-4c2b-bbc9-f501ed31f756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:32.307269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe82b23-680e-4c2b-bbc9-f501ed31f756 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:32.307951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe82b23-680e-4c2b-bbc9-f501ed31f756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:32.310220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe82b23-680e-4c2b-bbc9-f501ed31f756 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:32.333281 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1313/2625] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:32 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:32.337705 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d823f6-dbc5-48cc-857c-247306762c7e admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:32.348391 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d823f6-dbc5-48cc-857c-247306762c7e admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:32.353142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea41af0-fc51-4d34-85e4-423b90522283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:32.353744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea41af0-fc51-4d34-85e4-423b90522283 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:32.354026 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea41af0-fc51-4d34-85e4-423b90522283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:32.354299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea41af0-fc51-4d34-85e4-423b90522283 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:32.354948 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea41af0-fc51-4d34-85e4-423b90522283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:32.356604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea41af0-fc51-4d34-85e4-423b90522283 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:32.367002 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1313/2626] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:32.371041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe82b23-680e-4c2b-bbc9-f501ed31f756 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:32.381141 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe82b23-680e-4c2b-bbc9-f501ed31f756 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:32.386750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31467d-77f3-41ae-bc31-703b2d86cbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:32.387398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31467d-77f3-41ae-bc31-703b2d86cbb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:32.387696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31467d-77f3-41ae-bc31-703b2d86cbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:32.387981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31467d-77f3-41ae-bc31-703b2d86cbb9 admin admin] PATH_INFO: `/v3/projects/b1b0290159fc4d6a9266fc704d4086c9/users/470ab58e1ece45a2b1f44dc8a17df9cf/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:32.404407 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31467d-77f3-41ae-bc31-703b2d86cbb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b0290159fc4d6a9266fc704d4086c9, user_id=470ab58e1ece45a2b1f44dc8a17df9cf, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:32.407074 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31467d-77f3-41ae-bc31-703b2d86cbb9 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:32.422046 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1314/2627] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:32 2022] PUT /identity/v3/projects/b1b0290159fc4d6a9266fc704d4086c9/users/470ab58e1ece45a2b1f44dc8a17df9cf/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:32.426170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea41af0-fc51-4d34-85e4-423b90522283 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:32.437055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea41af0-fc51-4d34-85e4-423b90522283 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:32.502800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e6217-0afb-4a78-91a4-7aff88e774e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:32.503362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e6217-0afb-4a78-91a4-7aff88e774e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:32.503694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e6217-0afb-4a78-91a4-7aff88e774e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:32.504002 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e6217-0afb-4a78-91a4-7aff88e774e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:32.504642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e6217-0afb-4a78-91a4-7aff88e774e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:32.506248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e6217-0afb-4a78-91a4-7aff88e774e5 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:32.516693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1314/2628] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:32.521383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31467d-77f3-41ae-bc31-703b2d86cbb9 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:32.534962 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31467d-77f3-41ae-bc31-703b2d86cbb9 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:32.540476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b32d2-4c9b-4441-b44d-48df9631126f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:32.541168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b32d2-4c9b-4441-b44d-48df9631126f admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:32.541445 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b32d2-4c9b-4441-b44d-48df9631126f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:32.541710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b32d2-4c9b-4441-b44d-48df9631126f admin admin] PATH_INFO: `/v3/projects/b1b0290159fc4d6a9266fc704d4086c9/users/470ab58e1ece45a2b1f44dc8a17df9cf/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:32.543722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b32d2-4c9b-4441-b44d-48df9631126f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b0290159fc4d6a9266fc704d4086c9, user_id=470ab58e1ece45a2b1f44dc8a17df9cf, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:32.546423 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b32d2-4c9b-4441-b44d-48df9631126f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:32.557072 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1315/2629] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:32 2022] PUT /identity/v3/projects/b1b0290159fc4d6a9266fc704d4086c9/users/470ab58e1ece45a2b1f44dc8a17df9cf/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:32.569209 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9c420-13b9-4685-b956-bc4cf1cf8f91 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:32.569495 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9c420-13b9-4685-b956-bc4cf1cf8f91 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:32.569770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9c420-13b9-4685-b956-bc4cf1cf8f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:32.598377 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-2ed9c420-13b9-4685-b956-bc4cf1cf8f91 None None] MFA Rules not processed for user `470ab58e1ece45a2b1f44dc8a17df9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:32.599632 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2ed9c420-13b9-4685-b956-bc4cf1cf8f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:32.618135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:32.674474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1315/2630] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:32.686275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b32d2-4c9b-4441-b44d-48df9631126f admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:32.688147 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-526b32d2-4c9b-4441-b44d-48df9631126f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:32.732071 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:32.751973 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b32d2-4c9b-4441-b44d-48df9631126f admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:32.756008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89766b-b6ee-428e-a64b-b256b13c5233 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:32.756555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89766b-b6ee-428e-a64b-b256b13c5233 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:32.756830 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89766b-b6ee-428e-a64b-b256b13c5233 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:32.757097 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89766b-b6ee-428e-a64b-b256b13c5233 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:32.766551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89766b-b6ee-428e-a64b-b256b13c5233 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:32.768043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89766b-b6ee-428e-a64b-b256b13c5233 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:32.778252 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1316/2631] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:32.808255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9c420-13b9-4685-b956-bc4cf1cf8f91 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:32.874672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9c420-13b9-4685-b956-bc4cf1cf8f91 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:32.881770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30815261-6b95-42bf-b799-6376b0c5ebb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:32.882657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30815261-6b95-42bf-b799-6376b0c5ebb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:32.883224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30815261-6b95-42bf-b799-6376b0c5ebb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:32.883519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30815261-6b95-42bf-b799-6376b0c5ebb2 admin admin] PATH_INFO: `/v3/users/a5b07dab746341caadb5624137190975` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:32.884833 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30815261-6b95-42bf-b799-6376b0c5ebb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5b07dab746341caadb5624137190975)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:32.887191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30815261-6b95-42bf-b799-6376b0c5ebb2 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:32.938202 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-30815261-6b95-42bf-b799-6376b0c5ebb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5b07dab746341caadb5624137190975'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:32.946067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-30815261-6b95-42bf-b799-6376b0c5ebb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5b07dab746341caadb5624137190975'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:32.946819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-30815261-6b95-42bf-b799-6376b0c5ebb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5b07dab746341caadb5624137190975'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:32.947481 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-30815261-6b95-42bf-b799-6376b0c5ebb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5b07dab746341caadb5624137190975'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:32.960323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-30815261-6b95-42bf-b799-6376b0c5ebb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5b07dab746341caadb5624137190975'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:32.972644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1316/2632] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:32 2022] DELETE /identity/v3/users/a5b07dab746341caadb5624137190975 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:32.977089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e42f2-4366-4dfa-9f6f-f89ed452e0d6 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:32.977555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e42f2-4366-4dfa-9f6f-f89ed452e0d6 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:32.977653 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4e42f2-4366-4dfa-9f6f-f89ed452e0d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:32.996456 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-8d4e42f2-4366-4dfa-9f6f-f89ed452e0d6 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:32.997648 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-8d4e42f2-4366-4dfa-9f6f-f89ed452e0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:33.056690 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1317/2633] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:33.067337 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30815261-6b95-42bf-b799-6376b0c5ebb2 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:33.069055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-30815261-6b95-42bf-b799-6376b0c5ebb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:33.111868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:33.131579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30815261-6b95-42bf-b799-6376b0c5ebb2 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:33.135471 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285eab03-2a55-45dc-b7ff-906895886491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:33.136081 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285eab03-2a55-45dc-b7ff-906895886491 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:33.136461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285eab03-2a55-45dc-b7ff-906895886491 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:33.136656 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285eab03-2a55-45dc-b7ff-906895886491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:33.145952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285eab03-2a55-45dc-b7ff-906895886491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:33.147432 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285eab03-2a55-45dc-b7ff-906895886491 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:33.158340 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1317/2634] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:33.442225 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e42f2-4366-4dfa-9f6f-f89ed452e0d6 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:33.443679 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-8d4e42f2-4366-4dfa-9f6f-f89ed452e0d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:33.456317 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4e42f2-4366-4dfa-9f6f-f89ed452e0d6 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:33.461512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8eb882-be12-4813-9fe4-82b4d4283934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:33.462159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8eb882-be12-4813-9fe4-82b4d4283934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:33.462435 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8eb882-be12-4813-9fe4-82b4d4283934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:33.462705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8eb882-be12-4813-9fe4-82b4d4283934 admin admin] PATH_INFO: `/v3/projects/03fd5530dc0e40d386aebc3a185ec633` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:33.463917 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8eb882-be12-4813-9fe4-82b4d4283934 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03fd5530dc0e40d386aebc3a185ec633)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:33.465879 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8eb882-be12-4813-9fe4-82b4d4283934 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:33.515207 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-fa8eb882-be12-4813-9fe4-82b4d4283934 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03fd5530dc0e40d386aebc3a185ec633', 'request_id': 'req-fa8eb882-be12-4813-9fe4-82b4d4283934'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:33.524869 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-fa8eb882-be12-4813-9fe4-82b4d4283934 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03fd5530dc0e40d386aebc3a185ec633', 'request_id': 'req-fa8eb882-be12-4813-9fe4-82b4d4283934'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:33.530771 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-fa8eb882-be12-4813-9fe4-82b4d4283934 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03fd5530dc0e40d386aebc3a185ec633', 'request_id': 'req-fa8eb882-be12-4813-9fe4-82b4d4283934'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:33.542717 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-fa8eb882-be12-4813-9fe4-82b4d4283934 admin admin] The token cache is being invalidate because project 03fd5530dc0e40d386aebc3a185ec633 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:33.543049 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-fa8eb882-be12-4813-9fe4-82b4d4283934 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:33.543721 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-fa8eb882-be12-4813-9fe4-82b4d4283934 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:33.545420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1318/2635] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:33 2022] DELETE /identity/v3/projects/03fd5530dc0e40d386aebc3a185ec633 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:33.549659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285eab03-2a55-45dc-b7ff-906895886491 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:33.560473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285eab03-2a55-45dc-b7ff-906895886491 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:33.621804 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c41585-60bb-4577-9d40-0a4e5bf93667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:33.622439 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c41585-60bb-4577-9d40-0a4e5bf93667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:33.622715 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c41585-60bb-4577-9d40-0a4e5bf93667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:33.623003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c41585-60bb-4577-9d40-0a4e5bf93667 admin admin] PATH_INFO: `/v3/users/470ab58e1ece45a2b1f44dc8a17df9cf` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:33.624167 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c41585-60bb-4577-9d40-0a4e5bf93667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=470ab58e1ece45a2b1f44dc8a17df9cf)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:33.626091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c41585-60bb-4577-9d40-0a4e5bf93667 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:33.682312 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-57c41585-60bb-4577-9d40-0a4e5bf93667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '470ab58e1ece45a2b1f44dc8a17df9cf'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:33.691888 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-57c41585-60bb-4577-9d40-0a4e5bf93667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '470ab58e1ece45a2b1f44dc8a17df9cf'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:33.692539 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-57c41585-60bb-4577-9d40-0a4e5bf93667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '470ab58e1ece45a2b1f44dc8a17df9cf'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:33.693200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-57c41585-60bb-4577-9d40-0a4e5bf93667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '470ab58e1ece45a2b1f44dc8a17df9cf'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:33.706167 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-57c41585-60bb-4577-9d40-0a4e5bf93667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '470ab58e1ece45a2b1f44dc8a17df9cf'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:33.719740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1318/2636] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:33 2022] DELETE /identity/v3/users/470ab58e1ece45a2b1f44dc8a17df9cf => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:34.010536 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8eb882-be12-4813-9fe4-82b4d4283934 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:34.012053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-fa8eb882-be12-4813-9fe4-82b4d4283934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:34.025094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8eb882-be12-4813-9fe4-82b4d4283934 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:34.092236 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4c150-f2e0-4369-9663-6681b234d920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:34.092958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4c150-f2e0-4369-9663-6681b234d920 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.093237 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4c150-f2e0-4369-9663-6681b234d920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.093592 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4c150-f2e0-4369-9663-6681b234d920 admin admin] PATH_INFO: `/v3/projects/b1b0290159fc4d6a9266fc704d4086c9` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.094802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4c150-f2e0-4369-9663-6681b234d920 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b0290159fc4d6a9266fc704d4086c9)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:34.096705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4c150-f2e0-4369-9663-6681b234d920 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:34.099311 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:34.148241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f6a4c150-f2e0-4369-9663-6681b234d920 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b0290159fc4d6a9266fc704d4086c9', 'request_id': 'req-f6a4c150-f2e0-4369-9663-6681b234d920'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:34.156061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f6a4c150-f2e0-4369-9663-6681b234d920 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b0290159fc4d6a9266fc704d4086c9', 'request_id': 'req-f6a4c150-f2e0-4369-9663-6681b234d920'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:34.162182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f6a4c150-f2e0-4369-9663-6681b234d920 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b0290159fc4d6a9266fc704d4086c9', 'request_id': 'req-f6a4c150-f2e0-4369-9663-6681b234d920'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:34.173975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f6a4c150-f2e0-4369-9663-6681b234d920 admin admin] The token cache is being invalidate because project b1b0290159fc4d6a9266fc704d4086c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:34.174267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f6a4c150-f2e0-4369-9663-6681b234d920 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:34.174921 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f6a4c150-f2e0-4369-9663-6681b234d920 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:34.176506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1319/2637] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:34 2022] DELETE /identity/v3/projects/b1b0290159fc4d6a9266fc704d4086c9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:34.183367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce66cf-a18c-4955-91a9-e3eefd7fce81 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.183857 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce66cf-a18c-4955-91a9-e3eefd7fce81 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.183958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce66cf-a18c-4955-91a9-e3eefd7fce81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.205261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-3cce66cf-a18c-4955-91a9-e3eefd7fce81 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:34.206567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3cce66cf-a18c-4955-91a9-e3eefd7fce81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:34.256969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:34.274204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1319/2638] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:34.276345 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269f9fb-0c58-4277-a605-82841319e4b4 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.276687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269f9fb-0c58-4277-a605-82841319e4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.277040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269f9fb-0c58-4277-a605-82841319e4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.300152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-8269f9fb-0c58-4277-a605-82841319e4b4 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:34.300152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-8269f9fb-0c58-4277-a605-82841319e4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:34.307539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1320/2639] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:34.312145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce66cf-a18c-4955-91a9-e3eefd7fce81 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:34.321903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce66cf-a18c-4955-91a9-e3eefd7fce81 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:34.327054 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d97bd-a842-441e-aada-ccd364fbe688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:34.327716 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d97bd-a842-441e-aada-ccd364fbe688 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.328024 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d97bd-a842-441e-aada-ccd364fbe688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.328257 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d97bd-a842-441e-aada-ccd364fbe688 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.328912 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d97bd-a842-441e-aada-ccd364fbe688 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:15:34.329215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d97bd-a842-441e-aada-ccd364fbe688 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:34.330822 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d97bd-a842-441e-aada-ccd364fbe688 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:34.345780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1320/2640] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:15:34.346970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269f9fb-0c58-4277-a605-82841319e4b4 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:34.356492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269f9fb-0c58-4277-a605-82841319e4b4 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:34.361893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9043afe-5569-4823-8165-026f04f6c22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:34.362559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9043afe-5569-4823-8165-026f04f6c22f admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.363058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9043afe-5569-4823-8165-026f04f6c22f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.363333 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9043afe-5569-4823-8165-026f04f6c22f admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.364010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9043afe-5569-4823-8165-026f04f6c22f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:34.367489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9043afe-5569-4823-8165-026f04f6c22f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:34.396515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1321/2641] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:34 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:34.400941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d97bd-a842-441e-aada-ccd364fbe688 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:34.411009 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d97bd-a842-441e-aada-ccd364fbe688 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:34.475926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9043afe-5569-4823-8165-026f04f6c22f admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:34.477211 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c9043afe-5569-4823-8165-026f04f6c22f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:34.483252 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e63da-f29c-439d-986c-6638f31a4588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:34.484029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e63da-f29c-439d-986c-6638f31a4588 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.484402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e63da-f29c-439d-986c-6638f31a4588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.484769 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e63da-f29c-439d-986c-6638f31a4588 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.485513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e63da-f29c-439d-986c-6638f31a4588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:34.487565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e63da-f29c-439d-986c-6638f31a4588 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:34.518022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1321/2642] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:34 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:34.523520 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e63da-f29c-439d-986c-6638f31a4588 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:34.535783 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e63da-f29c-439d-986c-6638f31a4588 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:34.544881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aea87-ddac-440c-adbc-16ad22cfb214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:34.545711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aea87-ddac-440c-adbc-16ad22cfb214 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.546156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aea87-ddac-440c-adbc-16ad22cfb214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.546603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44aea87-ddac-440c-adbc-16ad22cfb214 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.547228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44aea87-ddac-440c-adbc-16ad22cfb214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:34.549248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44aea87-ddac-440c-adbc-16ad22cfb214 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:34.565981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1322/2643] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:34.570055 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aea87-ddac-440c-adbc-16ad22cfb214 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:34.572006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9043afe-5569-4823-8165-026f04f6c22f admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:34.578676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31a378-2cf1-4541-9a9b-5c8c97cf0588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:34.579436 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31a378-2cf1-4541-9a9b-5c8c97cf0588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.591976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44aea87-ddac-440c-adbc-16ad22cfb214 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:34.592096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31a378-2cf1-4541-9a9b-5c8c97cf0588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.592096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31a378-2cf1-4541-9a9b-5c8c97cf0588 admin admin] PATH_INFO: `/v3/users/6d2416bd945d4acd9bb263cab2223f3e` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.592096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31a378-2cf1-4541-9a9b-5c8c97cf0588 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d2416bd945d4acd9bb263cab2223f3e)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:34.592096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31a378-2cf1-4541-9a9b-5c8c97cf0588 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:34.592450 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7f597-65f6-48fb-979d-b545aa4dece2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:34.592450 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7f597-65f6-48fb-979d-b545aa4dece2 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.592450 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7f597-65f6-48fb-979d-b545aa4dece2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.592450 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d7f597-65f6-48fb-979d-b545aa4dece2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.592821 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7f597-65f6-48fb-979d-b545aa4dece2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:34.592821 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d7f597-65f6-48fb-979d-b545aa4dece2 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:34.602073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1323/2644] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:34.606211 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7f597-65f6-48fb-979d-b545aa4dece2 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:34.617598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d7f597-65f6-48fb-979d-b545aa4dece2 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:34.622908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dd5dba-b9b4-4c99-9260-60290b4b49d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:34.623570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd5dba-b9b4-4c99-9260-60290b4b49d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.623848 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd5dba-b9b4-4c99-9260-60290b4b49d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.624123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dd5dba-b9b4-4c99-9260-60290b4b49d2 admin admin] PATH_INFO: `/v3/projects/4cd8a5057ba24cb78e6d13fd5493ba26/users/6fa287f5517346dc9abacf6dc2d32141/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.635477 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bd31a378-2cf1-4541-9a9b-5c8c97cf0588 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d2416bd945d4acd9bb263cab2223f3e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:34.640738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd5dba-b9b4-4c99-9260-60290b4b49d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cd8a5057ba24cb78e6d13fd5493ba26, user_id=6fa287f5517346dc9abacf6dc2d32141, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:34.643791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bd31a378-2cf1-4541-9a9b-5c8c97cf0588 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d2416bd945d4acd9bb263cab2223f3e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:34.643991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dd5dba-b9b4-4c99-9260-60290b4b49d2 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:34.644832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bd31a378-2cf1-4541-9a9b-5c8c97cf0588 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d2416bd945d4acd9bb263cab2223f3e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:34.645468 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bd31a378-2cf1-4541-9a9b-5c8c97cf0588 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d2416bd945d4acd9bb263cab2223f3e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:34.653837 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-bd31a378-2cf1-4541-9a9b-5c8c97cf0588 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d2416bd945d4acd9bb263cab2223f3e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:34.655791 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:34.658409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1324/2645] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:34 2022] PUT /identity/v3/projects/4cd8a5057ba24cb78e6d13fd5493ba26/users/6fa287f5517346dc9abacf6dc2d32141/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:34.669540 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc12e0-5610-4d29-917c-c3673f6aa69a None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.669847 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc12e0-5610-4d29-917c-c3673f6aa69a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.670113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedc12e0-5610-4d29-917c-c3673f6aa69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.678995 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1322/2646] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:34 2022] DELETE /identity/v3/users/6d2416bd945d4acd9bb263cab2223f3e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:34.684045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d117826-7120-4529-abdc-d96e9c861e9d None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.684854 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d117826-7120-4529-abdc-d96e9c861e9d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.685176 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d117826-7120-4529-abdc-d96e9c861e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.702072 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-bedc12e0-5610-4d29-917c-c3673f6aa69a None None] MFA Rules not processed for user `6fa287f5517346dc9abacf6dc2d32141`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:34.703299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-bedc12e0-5610-4d29-917c-c3673f6aa69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:34.708347 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-4d117826-7120-4529-abdc-d96e9c861e9d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:34.709943 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4d117826-7120-4529-abdc-d96e9c861e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:34.773460 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1325/2647] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:34.776730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1323/2648] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:34.778184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc12e0-5610-4d29-917c-c3673f6aa69a None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:34.779981 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-bedc12e0-5610-4d29-917c-c3673f6aa69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:34.786316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d117826-7120-4529-abdc-d96e9c861e9d None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:34.787666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4d117826-7120-4529-abdc-d96e9c861e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:34.794010 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedc12e0-5610-4d29-917c-c3673f6aa69a None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:34.799325 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e059893-5208-473d-8189-2fca7aa91ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:34.800040 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e059893-5208-473d-8189-2fca7aa91ead admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.800244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e059893-5208-473d-8189-2fca7aa91ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.800514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e059893-5208-473d-8189-2fca7aa91ead admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.801160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e059893-5208-473d-8189-2fca7aa91ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:34.803092 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e059893-5208-473d-8189-2fca7aa91ead admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:34.819563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:34.833425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1326/2649] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:34 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:34.837701 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e059893-5208-473d-8189-2fca7aa91ead admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:34.849481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d117826-7120-4529-abdc-d96e9c861e9d None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:34.899224 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d76f0d-0b20-4df3-8a36-69a0a3310fb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:34.899866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d76f0d-0b20-4df3-8a36-69a0a3310fb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.900145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d76f0d-0b20-4df3-8a36-69a0a3310fb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.900535 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d76f0d-0b20-4df3-8a36-69a0a3310fb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.900653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e059893-5208-473d-8189-2fca7aa91ead admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:34.905485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b1b7f-972c-42b2-bd06-845f4cb3ab9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:34.906113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b1b7f-972c-42b2-bd06-845f4cb3ab9f admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.906380 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b1b7f-972c-42b2-bd06-845f4cb3ab9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.906646 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b1b7f-972c-42b2-bd06-845f4cb3ab9f admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.907285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b1b7f-972c-42b2-bd06-845f4cb3ab9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:34.909260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9b1b7f-972c-42b2-bd06-845f4cb3ab9f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:34.910403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d76f0d-0b20-4df3-8a36-69a0a3310fb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:34.911945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d76f0d-0b20-4df3-8a36-69a0a3310fb7 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:34.922197 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1324/2650] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:15:34.930576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1327/2651] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:34 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:34.934984 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d76f0d-0b20-4df3-8a36-69a0a3310fb7 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:34.944934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d76f0d-0b20-4df3-8a36-69a0a3310fb7 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:34.949586 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c8b85-cc66-4eb2-9e16-27aeefffc200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:34.950180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c8b85-cc66-4eb2-9e16-27aeefffc200 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.950458 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c8b85-cc66-4eb2-9e16-27aeefffc200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.950727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c8b85-cc66-4eb2-9e16-27aeefffc200 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.951437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c8b85-cc66-4eb2-9e16-27aeefffc200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:34.953053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c8b85-cc66-4eb2-9e16-27aeefffc200 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:34.963250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1325/2652] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:34.967247 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b1b7f-972c-42b2-bd06-845f4cb3ab9f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:34.978008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b1b7f-972c-42b2-bd06-845f4cb3ab9f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:34.984344 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d8c62-ed8a-4fa8-a1ad-7fcd06149224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:34.984985 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d8c62-ed8a-4fa8-a1ad-7fcd06149224 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:34.985273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d8c62-ed8a-4fa8-a1ad-7fcd06149224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:34.985502 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d8c62-ed8a-4fa8-a1ad-7fcd06149224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:34.986177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d8c62-ed8a-4fa8-a1ad-7fcd06149224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:34.987894 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d8c62-ed8a-4fa8-a1ad-7fcd06149224 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:35.006025 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1328/2653] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:35.007554 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c8b85-cc66-4eb2-9e16-27aeefffc200 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:35.018893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c8b85-cc66-4eb2-9e16-27aeefffc200 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:35.025282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cb822-1412-4fa1-b52d-b3223b207185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:35.025958 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cb822-1412-4fa1-b52d-b3223b207185 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.026234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cb822-1412-4fa1-b52d-b3223b207185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.026505 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67cb822-1412-4fa1-b52d-b3223b207185 admin admin] PATH_INFO: `/v3/projects/35bd47b95d534d7fbbcf744f79767565/users/641b83c56900415383504b1bdbc6c9ab/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.043505 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67cb822-1412-4fa1-b52d-b3223b207185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35bd47b95d534d7fbbcf744f79767565, user_id=641b83c56900415383504b1bdbc6c9ab, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:35.046348 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67cb822-1412-4fa1-b52d-b3223b207185 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:35.061817 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1326/2654] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:35 2022] PUT /identity/v3/projects/35bd47b95d534d7fbbcf744f79767565/users/641b83c56900415383504b1bdbc6c9ab/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:35.073325 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44ecc2-8f62-47f8-8b77-a6b54472d837 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.073609 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44ecc2-8f62-47f8-8b77-a6b54472d837 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.073890 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b44ecc2-8f62-47f8-8b77-a6b54472d837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.104680 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-2b44ecc2-8f62-47f8-8b77-a6b54472d837 None None] MFA Rules not processed for user `641b83c56900415383504b1bdbc6c9ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:35.105845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2b44ecc2-8f62-47f8-8b77-a6b54472d837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:35.173085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1329/2655] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:35.177756 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cb822-1412-4fa1-b52d-b3223b207185 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:35.188909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:35.211983 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44ecc2-8f62-47f8-8b77-a6b54472d837 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:35.213478 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2b44ecc2-8f62-47f8-8b77-a6b54472d837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:35.260898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67cb822-1412-4fa1-b52d-b3223b207185 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:35.265686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c63d83-f393-45e0-bccb-26af49f64f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:35.266295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c63d83-f393-45e0-bccb-26af49f64f0b admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.266573 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c63d83-f393-45e0-bccb-26af49f64f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.266838 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c63d83-f393-45e0-bccb-26af49f64f0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.267544 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c63d83-f393-45e0-bccb-26af49f64f0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:35.269476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c63d83-f393-45e0-bccb-26af49f64f0b admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:35.288725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b44ecc2-8f62-47f8-8b77-a6b54472d837 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:35.294232 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5ca6c9-f118-4c05-8b41-76d618966973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:35.294970 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ca6c9-f118-4c05-8b41-76d618966973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.295241 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ca6c9-f118-4c05-8b41-76d618966973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.295551 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5ca6c9-f118-4c05-8b41-76d618966973 admin admin] PATH_INFO: `/v3/projects/f079600e361845c6aac08d07151d1dea` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.296675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1327/2656] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:35 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:35.296812 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ca6c9-f118-4c05-8b41-76d618966973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f079600e361845c6aac08d07151d1dea)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:35.300759 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c63d83-f393-45e0-bccb-26af49f64f0b admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:35.301661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5ca6c9-f118-4c05-8b41-76d618966973 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:35.309663 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:35.358684 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-5d5ca6c9-f118-4c05-8b41-76d618966973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f079600e361845c6aac08d07151d1dea', 'request_id': 'req-5d5ca6c9-f118-4c05-8b41-76d618966973'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:35.370162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-5d5ca6c9-f118-4c05-8b41-76d618966973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f079600e361845c6aac08d07151d1dea', 'request_id': 'req-5d5ca6c9-f118-4c05-8b41-76d618966973'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:35.372112 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c63d83-f393-45e0-bccb-26af49f64f0b admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:35.376346 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-5d5ca6c9-f118-4c05-8b41-76d618966973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f079600e361845c6aac08d07151d1dea', 'request_id': 'req-5d5ca6c9-f118-4c05-8b41-76d618966973'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:35.388498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-5d5ca6c9-f118-4c05-8b41-76d618966973 admin admin] The token cache is being invalidate because project f079600e361845c6aac08d07151d1dea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:35.388840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-5d5ca6c9-f118-4c05-8b41-76d618966973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:35.389852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-5d5ca6c9-f118-4c05-8b41-76d618966973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:35.391581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1330/2657] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:35 2022] DELETE /identity/v3/projects/f079600e361845c6aac08d07151d1dea => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:35.396811 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05f86e-cfe1-46e5-948e-e25aca0e9931 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.397093 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05f86e-cfe1-46e5-948e-e25aca0e9931 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.397866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05f86e-cfe1-46e5-948e-e25aca0e9931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.417003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-8e05f86e-cfe1-46e5-948e-e25aca0e9931 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:35.418298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-8e05f86e-cfe1-46e5-948e-e25aca0e9931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:35.432096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ab868-3fdf-4f8b-bac1-c60b5854d5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:35.432716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ab868-3fdf-4f8b-bac1-c60b5854d5c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.432996 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ab868-3fdf-4f8b-bac1-c60b5854d5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.433270 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ab868-3fdf-4f8b-bac1-c60b5854d5c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.433908 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ab868-3fdf-4f8b-bac1-c60b5854d5c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:35.436406 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ab868-3fdf-4f8b-bac1-c60b5854d5c1 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:35.459954 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1328/2658] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:35 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:35.465464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ab868-3fdf-4f8b-bac1-c60b5854d5c1 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:35.467269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-8f0ab868-3fdf-4f8b-bac1-c60b5854d5c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:35.479132 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ab868-3fdf-4f8b-bac1-c60b5854d5c1 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:35.481235 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1331/2659] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:35.484099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e24fc-2242-49f6-8e78-14a5fe846c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:35.484700 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e24fc-2242-49f6-8e78-14a5fe846c28 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.484976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e24fc-2242-49f6-8e78-14a5fe846c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.485248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7e24fc-2242-49f6-8e78-14a5fe846c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.485895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e24fc-2242-49f6-8e78-14a5fe846c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:35.487556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7e24fc-2242-49f6-8e78-14a5fe846c28 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:35.490121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425c99-548c-4ba9-b6cc-c380cbd90fad None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.490415 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425c99-548c-4ba9-b6cc-c380cbd90fad None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.490982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19425c99-548c-4ba9-b6cc-c380cbd90fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.501595 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1329/2660] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:35.502659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e24fc-2242-49f6-8e78-14a5fe846c28 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:35.511595 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-19425c99-548c-4ba9-b6cc-c380cbd90fad None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:35.512846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-19425c99-548c-4ba9-b6cc-c380cbd90fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:35.513211 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7e24fc-2242-49f6-8e78-14a5fe846c28 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:35.518719 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9194236-5ebb-465d-91d9-96a767c999aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:35.519370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194236-5ebb-465d-91d9-96a767c999aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.519677 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194236-5ebb-465d-91d9-96a767c999aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.519953 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9194236-5ebb-465d-91d9-96a767c999aa admin admin] PATH_INFO: `/v3/projects/e8eb992df4d041ae9a4e44b5a8504503/users/564c8ea9c30c4ebf8f2e7cc8b3a13785/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.521120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1332/2661] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:35.525228 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19425c99-548c-4ba9-b6cc-c380cbd90fad None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:35.534170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19425c99-548c-4ba9-b6cc-c380cbd90fad None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:35.535727 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9194236-5ebb-465d-91d9-96a767c999aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8eb992df4d041ae9a4e44b5a8504503, user_id=564c8ea9c30c4ebf8f2e7cc8b3a13785, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:35.538338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9194236-5ebb-465d-91d9-96a767c999aa admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:35.538876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdafaf0-adcf-43ed-a24f-d48d14c17382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:35.539497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdafaf0-adcf-43ed-a24f-d48d14c17382 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.539780 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdafaf0-adcf-43ed-a24f-d48d14c17382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.540053 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdafaf0-adcf-43ed-a24f-d48d14c17382 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.540568 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:35.540687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdafaf0-adcf-43ed-a24f-d48d14c17382 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:15:35.540992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdafaf0-adcf-43ed-a24f-d48d14c17382 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:35.542634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdafaf0-adcf-43ed-a24f-d48d14c17382 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:35.554937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1330/2662] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:35 2022] PUT /identity/v3/projects/e8eb992df4d041ae9a4e44b5a8504503/users/564c8ea9c30c4ebf8f2e7cc8b3a13785/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:35.555312 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1333/2663] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:15:35.559892 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdafaf0-adcf-43ed-a24f-d48d14c17382 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:35.560051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9194236-5ebb-465d-91d9-96a767c999aa admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:35.570019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdafaf0-adcf-43ed-a24f-d48d14c17382 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:35.622429 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9194236-5ebb-465d-91d9-96a767c999aa admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:35.628249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906646d-3e2a-4245-86ef-8defdc682c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:35.628877 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906646d-3e2a-4245-86ef-8defdc682c52 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.629156 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906646d-3e2a-4245-86ef-8defdc682c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.629454 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906646d-3e2a-4245-86ef-8defdc682c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.630095 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906646d-3e2a-4245-86ef-8defdc682c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:35.632033 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906646d-3e2a-4245-86ef-8defdc682c52 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:35.634968 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e33b82-21b6-41a4-aabf-36bafd4b5f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:35.635776 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e33b82-21b6-41a4-aabf-36bafd4b5f1f admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.636056 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e33b82-21b6-41a4-aabf-36bafd4b5f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.636328 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e33b82-21b6-41a4-aabf-36bafd4b5f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.636977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e33b82-21b6-41a4-aabf-36bafd4b5f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:35.638657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e33b82-21b6-41a4-aabf-36bafd4b5f1f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:35.652792 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1334/2664] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:35.657251 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e33b82-21b6-41a4-aabf-36bafd4b5f1f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:35.664136 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1331/2665] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:35 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:35.668372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906646d-3e2a-4245-86ef-8defdc682c52 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:35.673219 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e33b82-21b6-41a4-aabf-36bafd4b5f1f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:35.680367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:35.745313 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906646d-3e2a-4245-86ef-8defdc682c52 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:35.749248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e814470-5755-4f30-9ef9-f78368cf0497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:35.750209 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8c5d5-e05f-4c3a-8621-648126deb490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:35.750824 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8c5d5-e05f-4c3a-8621-648126deb490 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.751047 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e814470-5755-4f30-9ef9-f78368cf0497 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.751158 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8c5d5-e05f-4c3a-8621-648126deb490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.751359 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8c5d5-e05f-4c3a-8621-648126deb490 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.751732 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e814470-5755-4f30-9ef9-f78368cf0497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.752164 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b8c5d5-e05f-4c3a-8621-648126deb490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:35.752288 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e814470-5755-4f30-9ef9-f78368cf0497 admin admin] PATH_INFO: `/v3/projects/e8eb992df4d041ae9a4e44b5a8504503/users/564c8ea9c30c4ebf8f2e7cc8b3a13785/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.754117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b8c5d5-e05f-4c3a-8621-648126deb490 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:35.754800 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e814470-5755-4f30-9ef9-f78368cf0497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8eb992df4d041ae9a4e44b5a8504503, user_id=564c8ea9c30c4ebf8f2e7cc8b3a13785, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:35.757674 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e814470-5755-4f30-9ef9-f78368cf0497 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:35.773764 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1335/2666] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:35 2022] PUT /identity/v3/projects/e8eb992df4d041ae9a4e44b5a8504503/users/564c8ea9c30c4ebf8f2e7cc8b3a13785/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:35.782764 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1332/2667] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:35 2022] POST /identity/v3/users => generated 284 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:35.783494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e814470-5755-4f30-9ef9-f78368cf0497 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:35.784381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807dbcc9-09e3-4b8d-af20-0655df8e08d4 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.784731 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807dbcc9-09e3-4b8d-af20-0655df8e08d4 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.785066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807dbcc9-09e3-4b8d-af20-0655df8e08d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.817794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-807dbcc9-09e3-4b8d-af20-0655df8e08d4 None None] MFA Rules not processed for user `564c8ea9c30c4ebf8f2e7cc8b3a13785`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:35.819385 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-807dbcc9-09e3-4b8d-af20-0655df8e08d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:35.850693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e814470-5755-4f30-9ef9-f78368cf0497 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:35.856128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf41cd-bfbe-40d7-b635-06d2c71a3551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:35.856945 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf41cd-bfbe-40d7-b635-06d2c71a3551 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.857566 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf41cd-bfbe-40d7-b635-06d2c71a3551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.857994 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf41cd-bfbe-40d7-b635-06d2c71a3551 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.858799 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf41cd-bfbe-40d7-b635-06d2c71a3551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:35.860606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bf41cd-bfbe-40d7-b635-06d2c71a3551 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:35.871172 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1336/2668] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:35.876431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf41cd-bfbe-40d7-b635-06d2c71a3551 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:35.883502 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:35.887409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf41cd-bfbe-40d7-b635-06d2c71a3551 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:35.892518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c4ae5-9686-49df-a8c4-7cadec48a0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:35.893171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c4ae5-9686-49df-a8c4-7cadec48a0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.893454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c4ae5-9686-49df-a8c4-7cadec48a0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.893675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70c4ae5-9686-49df-a8c4-7cadec48a0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.898709 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c4ae5-9686-49df-a8c4-7cadec48a0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:35.898884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1333/2669] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:35.898991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70c4ae5-9686-49df-a8c4-7cadec48a0e9 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:35.906654 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807dbcc9-09e3-4b8d-af20-0655df8e08d4 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:35.910265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1337/2670] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:35.910425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-807dbcc9-09e3-4b8d-af20-0655df8e08d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:35.911671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c4ae5-9686-49df-a8c4-7cadec48a0e9 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:35.921473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70c4ae5-9686-49df-a8c4-7cadec48a0e9 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:35.927592 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d15c3d-c5a2-4a8a-9b51-2f65fba70565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:35.928220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d15c3d-c5a2-4a8a-9b51-2f65fba70565 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.928826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d15c3d-c5a2-4a8a-9b51-2f65fba70565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.928826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d15c3d-c5a2-4a8a-9b51-2f65fba70565 admin admin] PATH_INFO: `/v3/projects/29210c432ea84bb0ae7135244d5b07e5/users/28212643f5c14ded9663ef33316965d6/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.946686 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d15c3d-c5a2-4a8a-9b51-2f65fba70565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29210c432ea84bb0ae7135244d5b07e5, user_id=28212643f5c14ded9663ef33316965d6, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:35.949371 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d15c3d-c5a2-4a8a-9b51-2f65fba70565 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:35.965372 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1338/2671] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:35 2022] PUT /identity/v3/projects/29210c432ea84bb0ae7135244d5b07e5/users/28212643f5c14ded9663ef33316965d6/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:35.976349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c05bd-12ac-412a-b16a-80f626a08491 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:35.976645 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c05bd-12ac-412a-b16a-80f626a08491 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:35.976910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c05bd-12ac-412a-b16a-80f626a08491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:35.978495 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807dbcc9-09e3-4b8d-af20-0655df8e08d4 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:36.019434 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-974c05bd-12ac-412a-b16a-80f626a08491 None None] MFA Rules not processed for user `28212643f5c14ded9663ef33316965d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:36.020872 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-974c05bd-12ac-412a-b16a-80f626a08491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:36.033516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb6172-0fc9-4372-8c6a-3a22eec6d9d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:36.034132 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb6172-0fc9-4372-8c6a-3a22eec6d9d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:36.034406 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb6172-0fc9-4372-8c6a-3a22eec6d9d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:36.034670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb6172-0fc9-4372-8c6a-3a22eec6d9d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:36.036526 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-77eb6172-0fc9-4372-8c6a-3a22eec6d9d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:36.081010 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:36.092868 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1339/2672] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:36.103895 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb6172-0fc9-4372-8c6a-3a22eec6d9d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:36.104262 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c05bd-12ac-412a-b16a-80f626a08491 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:36.105619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb6172-0fc9-4372-8c6a-3a22eec6d9d5 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:36.105754 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-974c05bd-12ac-412a-b16a-80f626a08491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:36.116077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1334/2673] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:15:36.117464 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c05bd-12ac-412a-b16a-80f626a08491 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:36.121265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c840e90-0ef3-4577-a95d-3bc633e1b257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:36.121821 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c840e90-0ef3-4577-a95d-3bc633e1b257 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:36.122212 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c840e90-0ef3-4577-a95d-3bc633e1b257 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:36.122355 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c840e90-0ef3-4577-a95d-3bc633e1b257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:36.131907 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c840e90-0ef3-4577-a95d-3bc633e1b257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:36.133607 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c840e90-0ef3-4577-a95d-3bc633e1b257 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:36.143059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1340/2674] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:15:36.861002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb6172-0fc9-4372-8c6a-3a22eec6d9d5 service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:36.862736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-77eb6172-0fc9-4372-8c6a-3a22eec6d9d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:36.931308 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb6172-0fc9-4372-8c6a-3a22eec6d9d5 service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:36.936822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf1a60-4b2a-416e-80f3-b5df4e3e6acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:36.937451 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf1a60-4b2a-416e-80f3-b5df4e3e6acc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:36.937720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf1a60-4b2a-416e-80f3-b5df4e3e6acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:36.937980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf1a60-4b2a-416e-80f3-b5df4e3e6acc admin admin] PATH_INFO: `/v3/users/d5be511ce1994a598756d63284a49846` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:36.939121 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf1a60-4b2a-416e-80f3-b5df4e3e6acc admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5be511ce1994a598756d63284a49846)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:36.941076 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf1a60-4b2a-416e-80f3-b5df4e3e6acc admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:36.945809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:36.990794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-79bf1a60-4b2a-416e-80f3-b5df4e3e6acc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5be511ce1994a598756d63284a49846'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:36.999437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-79bf1a60-4b2a-416e-80f3-b5df4e3e6acc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5be511ce1994a598756d63284a49846'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.000121 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-79bf1a60-4b2a-416e-80f3-b5df4e3e6acc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5be511ce1994a598756d63284a49846'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.000690 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-79bf1a60-4b2a-416e-80f3-b5df4e3e6acc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5be511ce1994a598756d63284a49846'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.007645 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-79bf1a60-4b2a-416e-80f3-b5df4e3e6acc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5be511ce1994a598756d63284a49846'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.024462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1335/2675] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:36 2022] DELETE /identity/v3/users/d5be511ce1994a598756d63284a49846 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:37.216440 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c840e90-0ef3-4577-a95d-3bc633e1b257 service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:37.227635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c840e90-0ef3-4577-a95d-3bc633e1b257 service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:37.291759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:37.292397 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:37.292708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:37.292977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec admin admin] PATH_INFO: `/v3/projects/81cbb2a3fa1541eebd180ba990e69838` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:37.294117 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec admin admin] RBAC: Authorizing `identity:delete_project(project_id=81cbb2a3fa1541eebd180ba990e69838)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:37.296152 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:37.350435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81cbb2a3fa1541eebd180ba990e69838', 'request_id': 'req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.359579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81cbb2a3fa1541eebd180ba990e69838', 'request_id': 'req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.365775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81cbb2a3fa1541eebd180ba990e69838', 'request_id': 'req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.377175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec admin admin] The token cache is being invalidate because project 81cbb2a3fa1541eebd180ba990e69838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:37.377457 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.378134 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.379640 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1341/2676] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:37 2022] DELETE /identity/v3/projects/81cbb2a3fa1541eebd180ba990e69838 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:37.383726 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf1a60-4b2a-416e-80f3-b5df4e3e6acc admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:37.385186 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-79bf1a60-4b2a-416e-80f3-b5df4e3e6acc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:37.398405 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf1a60-4b2a-416e-80f3-b5df4e3e6acc admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:37.460253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a8d13-2fc3-4296-b760-ebe94d22ce0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:37.460825 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a8d13-2fc3-4296-b760-ebe94d22ce0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:37.461105 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a8d13-2fc3-4296-b760-ebe94d22ce0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:37.461410 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2a8d13-2fc3-4296-b760-ebe94d22ce0f admin admin] PATH_INFO: `/v3/users/6650001c51db419dbfd9b3c7b1dd2467` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:37.462825 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a8d13-2fc3-4296-b760-ebe94d22ce0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6650001c51db419dbfd9b3c7b1dd2467)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:37.464978 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2a8d13-2fc3-4296-b760-ebe94d22ce0f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:37.522652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5b2a8d13-2fc3-4296-b760-ebe94d22ce0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6650001c51db419dbfd9b3c7b1dd2467'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.534577 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5b2a8d13-2fc3-4296-b760-ebe94d22ce0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6650001c51db419dbfd9b3c7b1dd2467'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.534577 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5b2a8d13-2fc3-4296-b760-ebe94d22ce0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6650001c51db419dbfd9b3c7b1dd2467'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.534861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5b2a8d13-2fc3-4296-b760-ebe94d22ce0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6650001c51db419dbfd9b3c7b1dd2467'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.543005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5b2a8d13-2fc3-4296-b760-ebe94d22ce0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6650001c51db419dbfd9b3c7b1dd2467'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.570940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1336/2677] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:37 2022] DELETE /identity/v3/users/6650001c51db419dbfd9b3c7b1dd2467 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:37.719037 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:37.720594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:37.772402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a8d13-2fc3-4296-b760-ebe94d22ce0f admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:37.774115 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5b2a8d13-2fc3-4296-b760-ebe94d22ce0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:37.779372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:37.805669 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c02eb-d664-4357-a0fa-a2f9bca1ffec admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:37.840776 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:37.864937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2a8d13-2fc3-4296-b760-ebe94d22ce0f admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:37.871299 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e173b-0eb9-41c6-b3ae-65d2e69b120d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:37.871981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e173b-0eb9-41c6-b3ae-65d2e69b120d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:37.872181 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1bf17-1fc2-447e-acdc-dbaaa92c879e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:37.872354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e173b-0eb9-41c6-b3ae-65d2e69b120d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:37.872600 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e173b-0eb9-41c6-b3ae-65d2e69b120d admin admin] PATH_INFO: `/v3/projects/f904fa9074924fadab446d6925f23596` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:37.873626 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1bf17-1fc2-447e-acdc-dbaaa92c879e service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:37.873745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e173b-0eb9-41c6-b3ae-65d2e69b120d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f904fa9074924fadab446d6925f23596)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:37.873934 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1bf17-1fc2-447e-acdc-dbaaa92c879e service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:37.874201 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1bf17-1fc2-447e-acdc-dbaaa92c879e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:37.875662 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567e173b-0eb9-41c6-b3ae-65d2e69b120d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:37.876056 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ffc1bf17-1fc2-447e-acdc-dbaaa92c879e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:37.944492 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-567e173b-0eb9-41c6-b3ae-65d2e69b120d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f904fa9074924fadab446d6925f23596', 'request_id': 'req-567e173b-0eb9-41c6-b3ae-65d2e69b120d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.954038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-567e173b-0eb9-41c6-b3ae-65d2e69b120d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f904fa9074924fadab446d6925f23596', 'request_id': 'req-567e173b-0eb9-41c6-b3ae-65d2e69b120d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.961257 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-567e173b-0eb9-41c6-b3ae-65d2e69b120d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f904fa9074924fadab446d6925f23596', 'request_id': 'req-567e173b-0eb9-41c6-b3ae-65d2e69b120d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.970051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1bf17-1fc2-447e-acdc-dbaaa92c879e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:37.971608 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1bf17-1fc2-447e-acdc-dbaaa92c879e service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:37.975582 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-567e173b-0eb9-41c6-b3ae-65d2e69b120d admin admin] The token cache is being invalidate because project f904fa9074924fadab446d6925f23596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:37.975872 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-567e173b-0eb9-41c6-b3ae-65d2e69b120d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.976523 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-567e173b-0eb9-41c6-b3ae-65d2e69b120d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:37.979486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1337/2678] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:37 2022] DELETE /identity/v3/projects/f904fa9074924fadab446d6925f23596 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:37.982183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1342/2679] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:37.983079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e51c7-ff97-406f-a9d0-90ad76e0dff5 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:37.983360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e51c7-ff97-406f-a9d0-90ad76e0dff5 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:37.983671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e51c7-ff97-406f-a9d0-90ad76e0dff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.004844 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-5a1e51c7-ff97-406f-a9d0-90ad76e0dff5 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:38.006214 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5a1e51c7-ff97-406f-a9d0-90ad76e0dff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:38.068537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1338/2680] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:38.077299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e44545-b280-4db9-946d-85b646022255 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.077584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e44545-b280-4db9-946d-85b646022255 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.079185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e44545-b280-4db9-946d-85b646022255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.102279 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-79e44545-b280-4db9-946d-85b646022255 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:38.104120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-79e44545-b280-4db9-946d-85b646022255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:38.113923 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1343/2681] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:38.118500 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e51c7-ff97-406f-a9d0-90ad76e0dff5 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:38.128351 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e51c7-ff97-406f-a9d0-90ad76e0dff5 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:38.133884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e22735-e062-45eb-956b-901d431c3df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:38.134486 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e22735-e062-45eb-956b-901d431c3df1 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.134759 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e22735-e062-45eb-956b-901d431c3df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.135024 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e22735-e062-45eb-956b-901d431c3df1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.136203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e22735-e062-45eb-956b-901d431c3df1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:15:38.136502 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e22735-e062-45eb-956b-901d431c3df1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:38.138596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e22735-e062-45eb-956b-901d431c3df1 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:38.150567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1339/2682] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:15:38.155176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e44545-b280-4db9-946d-85b646022255 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:38.165012 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e44545-b280-4db9-946d-85b646022255 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:38.169650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8c1e4-e717-4b81-9bec-a60a0eaf6ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:38.170327 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8c1e4-e717-4b81-9bec-a60a0eaf6ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.170606 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8c1e4-e717-4b81-9bec-a60a0eaf6ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.170874 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8c1e4-e717-4b81-9bec-a60a0eaf6ac3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.171553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8c1e4-e717-4b81-9bec-a60a0eaf6ac3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:38.173620 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8c1e4-e717-4b81-9bec-a60a0eaf6ac3 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:38.206173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1344/2683] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:38 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:38.210931 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e22735-e062-45eb-956b-901d431c3df1 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:38.221716 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e22735-e062-45eb-956b-901d431c3df1 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:38.243275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:38.286976 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfb2c5-0d99-4b9f-a962-eccfe6cc7a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:38.287625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfb2c5-0d99-4b9f-a962-eccfe6cc7a8e admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.287919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfb2c5-0d99-4b9f-a962-eccfe6cc7a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.288212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfb2c5-0d99-4b9f-a962-eccfe6cc7a8e admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.288858 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfb2c5-0d99-4b9f-a962-eccfe6cc7a8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:38.291152 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfb2c5-0d99-4b9f-a962-eccfe6cc7a8e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:38.316676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1340/2684] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:38 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:38.321106 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8c1e4-e717-4b81-9bec-a60a0eaf6ac3 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:38.334338 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8c1e4-e717-4b81-9bec-a60a0eaf6ac3 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:38.339390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb87c7f-58ba-44c1-9733-b3bd4b81bb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:38.340059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb87c7f-58ba-44c1-9733-b3bd4b81bb14 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.340336 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb87c7f-58ba-44c1-9733-b3bd4b81bb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.340596 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb87c7f-58ba-44c1-9733-b3bd4b81bb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.341254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb87c7f-58ba-44c1-9733-b3bd4b81bb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:38.343098 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb87c7f-58ba-44c1-9733-b3bd4b81bb14 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:38.355471 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1345/2685] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:38.359735 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfb2c5-0d99-4b9f-a962-eccfe6cc7a8e admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:38.372348 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfb2c5-0d99-4b9f-a962-eccfe6cc7a8e admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:38.377468 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e215cdb-9b3f-4970-9e67-1b12e6258270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:38.378072 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e215cdb-9b3f-4970-9e67-1b12e6258270 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.378349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e215cdb-9b3f-4970-9e67-1b12e6258270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.378619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e215cdb-9b3f-4970-9e67-1b12e6258270 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.379286 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e215cdb-9b3f-4970-9e67-1b12e6258270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:38.380963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e215cdb-9b3f-4970-9e67-1b12e6258270 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:38.392371 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1341/2686] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:38.397078 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb87c7f-58ba-44c1-9733-b3bd4b81bb14 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:38.407370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb87c7f-58ba-44c1-9733-b3bd4b81bb14 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:38.412215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed6cf4-69bb-43b2-bc60-4ce3b38a766b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:38.412941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed6cf4-69bb-43b2-bc60-4ce3b38a766b admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.413287 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed6cf4-69bb-43b2-bc60-4ce3b38a766b admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.413564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed6cf4-69bb-43b2-bc60-4ce3b38a766b admin admin] PATH_INFO: `/v3/projects/1f1dcbe5c54747c3bbc90344d17afc7e/users/3123f5fbc257437a852309db8c2364e5/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.428682 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed6cf4-69bb-43b2-bc60-4ce3b38a766b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f1dcbe5c54747c3bbc90344d17afc7e, user_id=3123f5fbc257437a852309db8c2364e5, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:38.431349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed6cf4-69bb-43b2-bc60-4ce3b38a766b admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:38.446099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1346/2687] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:38 2022] PUT /identity/v3/projects/1f1dcbe5c54747c3bbc90344d17afc7e/users/3123f5fbc257437a852309db8c2364e5/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:38.456245 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36c1e1-b86b-40f7-906c-0c2b6c393d09 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.456552 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36c1e1-b86b-40f7-906c-0c2b6c393d09 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.456863 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b36c1e1-b86b-40f7-906c-0c2b6c393d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.485501 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-5b36c1e1-b86b-40f7-906c-0c2b6c393d09 None None] MFA Rules not processed for user `3123f5fbc257437a852309db8c2364e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:38.486719 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5b36c1e1-b86b-40f7-906c-0c2b6c393d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:38.548256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed6cf4-69bb-43b2-bc60-4ce3b38a766b admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:38.549977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-1fed6cf4-69bb-43b2-bc60-4ce3b38a766b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:38.553149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1342/2688] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:38.557495 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36c1e1-b86b-40f7-906c-0c2b6c393d09 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:38.613281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed6cf4-69bb-43b2-bc60-4ce3b38a766b admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:38.617395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df10f8-183f-4f71-986e-506b400155e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:38.618003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df10f8-183f-4f71-986e-506b400155e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.618281 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df10f8-183f-4f71-986e-506b400155e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.618544 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df10f8-183f-4f71-986e-506b400155e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.620318 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d2df10f8-183f-4f71-986e-506b400155e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:38.620623 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b36c1e1-b86b-40f7-906c-0c2b6c393d09 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:38.625357 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ce467-9b9c-42db-8c92-c00119d637cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:38.625962 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ce467-9b9c-42db-8c92-c00119d637cb admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.626245 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ce467-9b9c-42db-8c92-c00119d637cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.626516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ce467-9b9c-42db-8c92-c00119d637cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.627277 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ce467-9b9c-42db-8c92-c00119d637cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:38.629210 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ce467-9b9c-42db-8c92-c00119d637cb admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:38.654518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:38.657635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1343/2689] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:38 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:38.661736 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ce467-9b9c-42db-8c92-c00119d637cb admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:38.675347 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:38.688895 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df10f8-183f-4f71-986e-506b400155e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:38.690344 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df10f8-183f-4f71-986e-506b400155e6 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:38.700260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1347/2690] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:38.726181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ce467-9b9c-42db-8c92-c00119d637cb admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:38.730796 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b353cde-79b2-45b4-b534-87f5b4fefe7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:38.731433 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b353cde-79b2-45b4-b534-87f5b4fefe7e admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.731710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b353cde-79b2-45b4-b534-87f5b4fefe7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.731975 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b353cde-79b2-45b4-b534-87f5b4fefe7e admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.732701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b353cde-79b2-45b4-b534-87f5b4fefe7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:38.734597 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b353cde-79b2-45b4-b534-87f5b4fefe7e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:38.760183 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1344/2691] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:38 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:38.761675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df10f8-183f-4f71-986e-506b400155e6 service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:38.772798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df10f8-183f-4f71-986e-506b400155e6 service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:38.777361 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045d540-9989-47f4-be9c-a3ae842503f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:38.777961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045d540-9989-47f4-be9c-a3ae842503f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.778231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045d540-9989-47f4-be9c-a3ae842503f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.778511 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e045d540-9989-47f4-be9c-a3ae842503f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.779321 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045d540-9989-47f4-be9c-a3ae842503f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:38.781468 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e045d540-9989-47f4-be9c-a3ae842503f5 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:38.792362 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1348/2692] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:38.796880 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b353cde-79b2-45b4-b534-87f5b4fefe7e admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:38.807473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b353cde-79b2-45b4-b534-87f5b4fefe7e admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:38.812704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cd0ee-a277-4f79-a759-d8082f17e0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:38.813408 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cd0ee-a277-4f79-a759-d8082f17e0aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.813680 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cd0ee-a277-4f79-a759-d8082f17e0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.813972 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cd0ee-a277-4f79-a759-d8082f17e0aa admin admin] PATH_INFO: `/v3/projects/014f48ee10a34ff6ba8b904e055a5d19/users/a0e7285b90c4435d821eb2a3c0a1642c/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.829610 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9cd0ee-a277-4f79-a759-d8082f17e0aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=014f48ee10a34ff6ba8b904e055a5d19, user_id=a0e7285b90c4435d821eb2a3c0a1642c, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:38.832428 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9cd0ee-a277-4f79-a759-d8082f17e0aa admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:38.847073 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1345/2693] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:38 2022] PUT /identity/v3/projects/014f48ee10a34ff6ba8b904e055a5d19/users/a0e7285b90c4435d821eb2a3c0a1642c/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:38.851394 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045d540-9989-47f4-be9c-a3ae842503f5 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:38.861908 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e045d540-9989-47f4-be9c-a3ae842503f5 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:38.919792 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee68a6-499b-4695-8b2d-2d2e929991a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:38.920404 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee68a6-499b-4695-8b2d-2d2e929991a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.920675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee68a6-499b-4695-8b2d-2d2e929991a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.920941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee68a6-499b-4695-8b2d-2d2e929991a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.921582 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee68a6-499b-4695-8b2d-2d2e929991a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:38.923188 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee68a6-499b-4695-8b2d-2d2e929991a8 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:38.933272 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1349/2694] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:38.939489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cd0ee-a277-4f79-a759-d8082f17e0aa admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:38.953026 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cd0ee-a277-4f79-a759-d8082f17e0aa admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:38.958506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402546f-ddbe-4a34-b530-305d27ec0f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:38.959167 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402546f-ddbe-4a34-b530-305d27ec0f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.959459 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402546f-ddbe-4a34-b530-305d27ec0f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.959749 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402546f-ddbe-4a34-b530-305d27ec0f96 admin admin] PATH_INFO: `/v3/projects/014f48ee10a34ff6ba8b904e055a5d19/users/a0e7285b90c4435d821eb2a3c0a1642c/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:38.961917 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402546f-ddbe-4a34-b530-305d27ec0f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=014f48ee10a34ff6ba8b904e055a5d19, user_id=a0e7285b90c4435d821eb2a3c0a1642c, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:38.964672 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402546f-ddbe-4a34-b530-305d27ec0f96 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:38.975763 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1346/2695] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:38 2022] PUT /identity/v3/projects/014f48ee10a34ff6ba8b904e055a5d19/users/a0e7285b90c4435d821eb2a3c0a1642c/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:38.988130 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c5f96-1009-4dc9-88c1-3fea2759f2c0 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:38.988483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c5f96-1009-4dc9-88c1-3fea2759f2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:38.988818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c5f96-1009-4dc9-88c1-3fea2759f2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:39.018965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-902c5f96-1009-4dc9-88c1-3fea2759f2c0 None None] MFA Rules not processed for user `a0e7285b90c4435d821eb2a3c0a1642c`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:39.020279 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-902c5f96-1009-4dc9-88c1-3fea2759f2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:39.098455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1350/2696] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:38 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:39.109248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402546f-ddbe-4a34-b530-305d27ec0f96 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:39.166052 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402546f-ddbe-4a34-b530-305d27ec0f96 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:39.170497 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9901b1df-8a32-4db9-82ca-a8621fe7dd6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:39.171161 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901b1df-8a32-4db9-82ca-a8621fe7dd6f service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:39.171437 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901b1df-8a32-4db9-82ca-a8621fe7dd6f service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:39.171734 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9901b1df-8a32-4db9-82ca-a8621fe7dd6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:39.230656 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901b1df-8a32-4db9-82ca-a8621fe7dd6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:39.232519 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9901b1df-8a32-4db9-82ca-a8621fe7dd6f service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:39.241864 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1347/2697] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:41.810914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c5f96-1009-4dc9-88c1-3fea2759f2c0 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:41.859813 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:41.880536 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c5f96-1009-4dc9-88c1-3fea2759f2c0 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:41.885265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2162d-a8e6-479e-8c99-55f9d23d1551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:41.885899 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2162d-a8e6-479e-8c99-55f9d23d1551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:41.886174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2162d-a8e6-479e-8c99-55f9d23d1551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:41.886439 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2162d-a8e6-479e-8c99-55f9d23d1551 admin admin] PATH_INFO: `/v3/users/3123f5fbc257437a852309db8c2364e5` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:41.887636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2162d-a8e6-479e-8c99-55f9d23d1551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3123f5fbc257437a852309db8c2364e5)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:41.890559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2162d-a8e6-479e-8c99-55f9d23d1551 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:41.941721 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0ee2162d-a8e6-479e-8c99-55f9d23d1551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3123f5fbc257437a852309db8c2364e5'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:41.950514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0ee2162d-a8e6-479e-8c99-55f9d23d1551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3123f5fbc257437a852309db8c2364e5'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:41.951226 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0ee2162d-a8e6-479e-8c99-55f9d23d1551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3123f5fbc257437a852309db8c2364e5'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:41.951864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0ee2162d-a8e6-479e-8c99-55f9d23d1551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3123f5fbc257437a852309db8c2364e5'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:41.963596 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-0ee2162d-a8e6-479e-8c99-55f9d23d1551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3123f5fbc257437a852309db8c2364e5'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:41.975319 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1351/2698] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:41 2022] DELETE /identity/v3/users/3123f5fbc257437a852309db8c2364e5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:41.979983 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a587b6d-f0bb-485a-9d9d-bf19fde4607d None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:41.980493 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a587b6d-f0bb-485a-9d9d-bf19fde4607d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:41.980493 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a587b6d-f0bb-485a-9d9d-bf19fde4607d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:42.000374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-7a587b6d-f0bb-485a-9d9d-bf19fde4607d None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:42.001571 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-7a587b6d-f0bb-485a-9d9d-bf19fde4607d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:42.015904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:42.064476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1348/2699] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:42.075917 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2162d-a8e6-479e-8c99-55f9d23d1551 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:42.077579 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0ee2162d-a8e6-479e-8c99-55f9d23d1551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:42.138942 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2162d-a8e6-479e-8c99-55f9d23d1551 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:42.143085 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719094dc-5a40-493d-a1b4-063320105229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:42.143718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719094dc-5a40-493d-a1b4-063320105229 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:42.144001 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719094dc-5a40-493d-a1b4-063320105229 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:42.144269 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719094dc-5a40-493d-a1b4-063320105229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:42.153821 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719094dc-5a40-493d-a1b4-063320105229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:42.155295 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719094dc-5a40-493d-a1b4-063320105229 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:42.165272 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1352/2700] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:42.444360 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a587b6d-f0bb-485a-9d9d-bf19fde4607d None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:42.446071 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-7a587b6d-f0bb-485a-9d9d-bf19fde4607d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:42.466059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a587b6d-f0bb-485a-9d9d-bf19fde4607d None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:42.478045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:42.478769 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:42.479043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:42.479375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae admin admin] PATH_INFO: `/v3/projects/1f1dcbe5c54747c3bbc90344d17afc7e` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:42.480681 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f1dcbe5c54747c3bbc90344d17afc7e)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:42.482771 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:42.538577 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f1dcbe5c54747c3bbc90344d17afc7e', 'request_id': 'req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:42.551010 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f1dcbe5c54747c3bbc90344d17afc7e', 'request_id': 'req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:42.557612 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f1dcbe5c54747c3bbc90344d17afc7e', 'request_id': 'req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:42.569613 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae admin admin] The token cache is being invalidate because project 1f1dcbe5c54747c3bbc90344d17afc7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:42.569924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:42.570578 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:42.572220 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1349/2701] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:42 2022] DELETE /identity/v3/projects/1f1dcbe5c54747c3bbc90344d17afc7e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:42.576423 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719094dc-5a40-493d-a1b4-063320105229 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:42.586043 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719094dc-5a40-493d-a1b4-063320105229 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:42.616902 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:42.618470 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:42.645737 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56bafe5-9517-4fad-a0bf-c879d1710cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:42.646384 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bafe5-9517-4fad-a0bf-c879d1710cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:42.646653 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bafe5-9517-4fad-a0bf-c879d1710cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:42.646918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56bafe5-9517-4fad-a0bf-c879d1710cdd admin admin] PATH_INFO: `/v3/users/a0e7285b90c4435d821eb2a3c0a1642c` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:42.648088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bafe5-9517-4fad-a0bf-c879d1710cdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0e7285b90c4435d821eb2a3c0a1642c)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:42.650023 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56bafe5-9517-4fad-a0bf-c879d1710cdd admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:42.652642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:42.685862 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a512a45-0ed6-4d7a-9ead-2a764f4294ae admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:42.690980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b875441-a4d8-432a-bed7-98818343d266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:42.691815 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b875441-a4d8-432a-bed7-98818343d266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:42.692087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b875441-a4d8-432a-bed7-98818343d266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:42.692362 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b875441-a4d8-432a-bed7-98818343d266 admin admin] PATH_INFO: `/v3/users/6fa287f5517346dc9abacf6dc2d32141` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:42.693498 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b875441-a4d8-432a-bed7-98818343d266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fa287f5517346dc9abacf6dc2d32141)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:42.695412 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b875441-a4d8-432a-bed7-98818343d266 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:42.705028 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a56bafe5-9517-4fad-a0bf-c879d1710cdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0e7285b90c4435d821eb2a3c0a1642c'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:42.713857 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a56bafe5-9517-4fad-a0bf-c879d1710cdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0e7285b90c4435d821eb2a3c0a1642c'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:42.714512 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a56bafe5-9517-4fad-a0bf-c879d1710cdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0e7285b90c4435d821eb2a3c0a1642c'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:42.715156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a56bafe5-9517-4fad-a0bf-c879d1710cdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0e7285b90c4435d821eb2a3c0a1642c'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:42.732873 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-a56bafe5-9517-4fad-a0bf-c879d1710cdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0e7285b90c4435d821eb2a3c0a1642c'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:42.746442 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1353/2702] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:42 2022] DELETE /identity/v3/users/a0e7285b90c4435d821eb2a3c0a1642c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:42.746946 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:42.748945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-3b875441-a4d8-432a-bed7-98818343d266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fa287f5517346dc9abacf6dc2d32141'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:42.757880 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-3b875441-a4d8-432a-bed7-98818343d266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fa287f5517346dc9abacf6dc2d32141'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:42.759017 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-3b875441-a4d8-432a-bed7-98818343d266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fa287f5517346dc9abacf6dc2d32141'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:42.759644 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-3b875441-a4d8-432a-bed7-98818343d266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fa287f5517346dc9abacf6dc2d32141'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:42.768284 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-3b875441-a4d8-432a-bed7-98818343d266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fa287f5517346dc9abacf6dc2d32141'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:42.787165 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1350/2703] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:42 2022] DELETE /identity/v3/users/6fa287f5517346dc9abacf6dc2d32141 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:42.791720 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567eb94-5cd7-4d15-ae49-1de4e3a3a06a None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:42.792028 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567eb94-5cd7-4d15-ae49-1de4e3a3a06a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:42.792330 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567eb94-5cd7-4d15-ae49-1de4e3a3a06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:42.812401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-c567eb94-5cd7-4d15-ae49-1de4e3a3a06a None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:42.813639 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c567eb94-5cd7-4d15-ae49-1de4e3a3a06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:42.875759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1354/2704] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:42.885825 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b875441-a4d8-432a-bed7-98818343d266 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:42.887249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3b875441-a4d8-432a-bed7-98818343d266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:42.954997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b875441-a4d8-432a-bed7-98818343d266 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:42.959666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfe465-404d-4ae8-b275-cfe5a7b027bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:42.960416 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfe465-404d-4ae8-b275-cfe5a7b027bc service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:42.960703 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfe465-404d-4ae8-b275-cfe5a7b027bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:42.960950 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfe465-404d-4ae8-b275-cfe5a7b027bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:42.972005 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfe465-404d-4ae8-b275-cfe5a7b027bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:42.973658 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbfe465-404d-4ae8-b275-cfe5a7b027bc service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:42.984397 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1351/2705] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:43.049475 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567eb94-5cd7-4d15-ae49-1de4e3a3a06a None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:43.051090 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c567eb94-5cd7-4d15-ae49-1de4e3a3a06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:43.064145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567eb94-5cd7-4d15-ae49-1de4e3a3a06a None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:43.071538 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9520633-2093-4e66-b637-8e4a8100aacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:43.082421 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9520633-2093-4e66-b637-8e4a8100aacc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:43.084168 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9520633-2093-4e66-b637-8e4a8100aacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:43.084518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9520633-2093-4e66-b637-8e4a8100aacc admin admin] PATH_INFO: `/v3/projects/014f48ee10a34ff6ba8b904e055a5d19` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:43.086241 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9520633-2093-4e66-b637-8e4a8100aacc admin admin] RBAC: Authorizing `identity:delete_project(project_id=014f48ee10a34ff6ba8b904e055a5d19)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:43.089842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9520633-2093-4e66-b637-8e4a8100aacc admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:43.142865 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c9520633-2093-4e66-b637-8e4a8100aacc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '014f48ee10a34ff6ba8b904e055a5d19', 'request_id': 'req-c9520633-2093-4e66-b637-8e4a8100aacc'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:43.153118 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c9520633-2093-4e66-b637-8e4a8100aacc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '014f48ee10a34ff6ba8b904e055a5d19', 'request_id': 'req-c9520633-2093-4e66-b637-8e4a8100aacc'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:43.160107 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c9520633-2093-4e66-b637-8e4a8100aacc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '014f48ee10a34ff6ba8b904e055a5d19', 'request_id': 'req-c9520633-2093-4e66-b637-8e4a8100aacc'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:43.172115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c9520633-2093-4e66-b637-8e4a8100aacc admin admin] The token cache is being invalidate because project 014f48ee10a34ff6ba8b904e055a5d19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:43.172412 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c9520633-2093-4e66-b637-8e4a8100aacc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:43.173109 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-c9520633-2093-4e66-b637-8e4a8100aacc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:43.174739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1355/2706] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:43 2022] DELETE /identity/v3/projects/014f48ee10a34ff6ba8b904e055a5d19 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:43.179805 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436ad54-9ac4-4f97-ba3d-cb2ffbd218c6 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:43.180102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436ad54-9ac4-4f97-ba3d-cb2ffbd218c6 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:43.180372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2436ad54-9ac4-4f97-ba3d-cb2ffbd218c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:43.201460 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-2436ad54-9ac4-4f97-ba3d-cb2ffbd218c6 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:43.202683 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-2436ad54-9ac4-4f97-ba3d-cb2ffbd218c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:43.262776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1352/2707] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:43.271885 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcba7c-156b-4774-b6dc-18504dd3f611 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:43.272203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcba7c-156b-4774-b6dc-18504dd3f611 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:43.272477 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bcba7c-156b-4774-b6dc-18504dd3f611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:43.282795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:43.296220 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-59bcba7c-156b-4774-b6dc-18504dd3f611 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:43.298564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-59bcba7c-156b-4774-b6dc-18504dd3f611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:43.309463 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1356/2708] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:43.310204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436ad54-9ac4-4f97-ba3d-cb2ffbd218c6 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:43.311977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-2436ad54-9ac4-4f97-ba3d-cb2ffbd218c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:43.313412 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcba7c-156b-4774-b6dc-18504dd3f611 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:43.329713 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bcba7c-156b-4774-b6dc-18504dd3f611 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:43.332814 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2436ad54-9ac4-4f97-ba3d-cb2ffbd218c6 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:43.335297 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a1b90-3d1e-4497-bad1-13a042982ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:43.336091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a1b90-3d1e-4497-bad1-13a042982ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:43.336526 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a1b90-3d1e-4497-bad1-13a042982ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:43.336903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a1b90-3d1e-4497-bad1-13a042982ac7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:43.337752 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a1b90-3d1e-4497-bad1-13a042982ac7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:15:43.338177 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a1b90-3d1e-4497-bad1-13a042982ac7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:43.338394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbbd71-4129-42ca-b366-9503d34cda98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:43.339060 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbbd71-4129-42ca-b366-9503d34cda98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:43.339334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbbd71-4129-42ca-b366-9503d34cda98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:43.339650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bbbd71-4129-42ca-b366-9503d34cda98 admin admin] PATH_INFO: `/v3/projects/4cd8a5057ba24cb78e6d13fd5493ba26` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:43.340171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89a1b90-3d1e-4497-bad1-13a042982ac7 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:43.340783 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbbd71-4129-42ca-b366-9503d34cda98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cd8a5057ba24cb78e6d13fd5493ba26)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:43.342717 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bbbd71-4129-42ca-b366-9503d34cda98 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:43.353639 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1357/2709] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:15:43.357922 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a1b90-3d1e-4497-bad1-13a042982ac7 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:43.367836 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a1b90-3d1e-4497-bad1-13a042982ac7 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:43.372913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b2fd6-074c-46a1-959c-c41fe0b26422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:43.373713 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b2fd6-074c-46a1-959c-c41fe0b26422 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:43.374074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b2fd6-074c-46a1-959c-c41fe0b26422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:43.374498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b2fd6-074c-46a1-959c-c41fe0b26422 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:43.375185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b2fd6-074c-46a1-959c-c41fe0b26422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:43.377173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65b2fd6-074c-46a1-959c-c41fe0b26422 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:43.391992 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-26bbbd71-4129-42ca-b366-9503d34cda98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cd8a5057ba24cb78e6d13fd5493ba26', 'request_id': 'req-26bbbd71-4129-42ca-b366-9503d34cda98'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:43.400637 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-26bbbd71-4129-42ca-b366-9503d34cda98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cd8a5057ba24cb78e6d13fd5493ba26', 'request_id': 'req-26bbbd71-4129-42ca-b366-9503d34cda98'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:43.404982 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:43.407120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1358/2710] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:43 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:43.407342 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-26bbbd71-4129-42ca-b366-9503d34cda98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cd8a5057ba24cb78e6d13fd5493ba26', 'request_id': 'req-26bbbd71-4129-42ca-b366-9503d34cda98'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:43.411253 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b2fd6-074c-46a1-959c-c41fe0b26422 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:43.421680 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-26bbbd71-4129-42ca-b366-9503d34cda98 admin admin] The token cache is being invalidate because project 4cd8a5057ba24cb78e6d13fd5493ba26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:43.421970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-26bbbd71-4129-42ca-b366-9503d34cda98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:43.422649 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-26bbbd71-4129-42ca-b366-9503d34cda98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:43.424372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1353/2711] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:43 2022] DELETE /identity/v3/projects/4cd8a5057ba24cb78e6d13fd5493ba26 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:43.428064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbbd71-4129-42ca-b366-9503d34cda98 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:43.429382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-26bbbd71-4129-42ca-b366-9503d34cda98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:43.472625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b2fd6-074c-46a1-959c-c41fe0b26422 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:43.477199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdf85d-2b30-4ea5-b638-a86c29b6539f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:43.477826 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdf85d-2b30-4ea5-b638-a86c29b6539f admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:43.478101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdf85d-2b30-4ea5-b638-a86c29b6539f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:43.478473 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cdf85d-2b30-4ea5-b638-a86c29b6539f admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:43.479123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdf85d-2b30-4ea5-b638-a86c29b6539f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:43.481088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cdf85d-2b30-4ea5-b638-a86c29b6539f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:43.492566 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bbbd71-4129-42ca-b366-9503d34cda98 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:43.498066 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee96744-2ee4-480b-bdb8-51693033a5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:43.498777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee96744-2ee4-480b-bdb8-51693033a5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:43.499096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee96744-2ee4-480b-bdb8-51693033a5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:43.499335 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee96744-2ee4-480b-bdb8-51693033a5b4 admin admin] PATH_INFO: `/v3/users/641b83c56900415383504b1bdbc6c9ab` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:43.500697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee96744-2ee4-480b-bdb8-51693033a5b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=641b83c56900415383504b1bdbc6c9ab)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:43.502610 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee96744-2ee4-480b-bdb8-51693033a5b4 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:43.508884 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1359/2712] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:43 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:43.509965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdf85d-2b30-4ea5-b638-a86c29b6539f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:43.511598 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-54cdf85d-2b30-4ea5-b638-a86c29b6539f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:43.523852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cdf85d-2b30-4ea5-b638-a86c29b6539f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:43.529334 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6a0f9-c319-4793-a8b4-70204277d5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:43.530008 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6a0f9-c319-4793-a8b4-70204277d5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:43.530272 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6a0f9-c319-4793-a8b4-70204277d5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:43.530527 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6a0f9-c319-4793-a8b4-70204277d5d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:43.531163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6a0f9-c319-4793-a8b4-70204277d5d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:43.532799 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c6a0f9-c319-4793-a8b4-70204277d5d9 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:43.543335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1360/2713] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:43.547342 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6a0f9-c319-4793-a8b4-70204277d5d9 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:43.557109 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6a0f9-c319-4793-a8b4-70204277d5d9 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:43.560279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2ee96744-2ee4-480b-bdb8-51693033a5b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '641b83c56900415383504b1bdbc6c9ab'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:43.562346 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ab41b-6a17-4b94-91c8-68eb81dbf70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:43.563007 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ab41b-6a17-4b94-91c8-68eb81dbf70d admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:43.563200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ab41b-6a17-4b94-91c8-68eb81dbf70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:43.563482 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ab41b-6a17-4b94-91c8-68eb81dbf70d admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:43.564170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ab41b-6a17-4b94-91c8-68eb81dbf70d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:43.565758 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ab41b-6a17-4b94-91c8-68eb81dbf70d admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:43.570629 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2ee96744-2ee4-480b-bdb8-51693033a5b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '641b83c56900415383504b1bdbc6c9ab'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:43.572370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2ee96744-2ee4-480b-bdb8-51693033a5b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '641b83c56900415383504b1bdbc6c9ab'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:43.573304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2ee96744-2ee4-480b-bdb8-51693033a5b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '641b83c56900415383504b1bdbc6c9ab'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:43.579619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1361/2714] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:43.580964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ab41b-6a17-4b94-91c8-68eb81dbf70d admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:43.583896 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-9b8ab41b-6a17-4b94-91c8-68eb81dbf70d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:43.585856 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2ee96744-2ee4-480b-bdb8-51693033a5b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '641b83c56900415383504b1bdbc6c9ab'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:43.597883 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ab41b-6a17-4b94-91c8-68eb81dbf70d admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:43.603043 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94f3f0-75a7-41e2-94dd-bd4fdd4e53bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:43.603781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f3f0-75a7-41e2-94dd-bd4fdd4e53bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:43.604087 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f3f0-75a7-41e2-94dd-bd4fdd4e53bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:43.604327 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f94f3f0-75a7-41e2-94dd-bd4fdd4e53bb admin admin] PATH_INFO: `/v3/projects/8a6ef62e146a45ecb2c1d0b179713ed5/users/bd619e168712480dac93519c4c469e71/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:43.605378 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1354/2715] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:43 2022] DELETE /identity/v3/users/641b83c56900415383504b1bdbc6c9ab => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:43.620550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f3f0-75a7-41e2-94dd-bd4fdd4e53bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a6ef62e146a45ecb2c1d0b179713ed5, user_id=bd619e168712480dac93519c4c469e71, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:43.623312 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f94f3f0-75a7-41e2-94dd-bd4fdd4e53bb admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:43.638602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1362/2716] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:43 2022] PUT /identity/v3/projects/8a6ef62e146a45ecb2c1d0b179713ed5/users/bd619e168712480dac93519c4c469e71/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:43.653697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec083c-8c01-4070-ba85-613e4d71123e None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:43.654785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec083c-8c01-4070-ba85-613e4d71123e None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:43.655387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec083c-8c01-4070-ba85-613e4d71123e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:43.689168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-46ec083c-8c01-4070-ba85-613e4d71123e None None] MFA Rules not processed for user `bd619e168712480dac93519c4c469e71`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:43.690636 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-46ec083c-8c01-4070-ba85-613e4d71123e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:43.766524 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:43.779833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1355/2717] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:43.787433 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94f3f0-75a7-41e2-94dd-bd4fdd4e53bb admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:43.789836 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-9f94f3f0-75a7-41e2-94dd-bd4fdd4e53bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:43.858775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f94f3f0-75a7-41e2-94dd-bd4fdd4e53bb admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:43.863141 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bf8eb-314b-428d-a6fc-826b5454e1ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:43.863788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bf8eb-314b-428d-a6fc-826b5454e1ea service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:43.864063 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bf8eb-314b-428d-a6fc-826b5454e1ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:43.864326 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641bf8eb-314b-428d-a6fc-826b5454e1ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:43.873852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bf8eb-314b-428d-a6fc-826b5454e1ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:43.875699 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641bf8eb-314b-428d-a6fc-826b5454e1ea service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:43.886596 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1363/2718] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:43.905622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec083c-8c01-4070-ba85-613e4d71123e None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:43.907087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-46ec083c-8c01-4070-ba85-613e4d71123e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:43.981077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec083c-8c01-4070-ba85-613e4d71123e None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:43.986396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9df87b-b3f8-43d1-8692-bfeafa80e646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:43.987068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9df87b-b3f8-43d1-8692-bfeafa80e646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:43.987337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9df87b-b3f8-43d1-8692-bfeafa80e646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:43.987642 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9df87b-b3f8-43d1-8692-bfeafa80e646 admin admin] PATH_INFO: `/v3/projects/35bd47b95d534d7fbbcf744f79767565` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:43.988786 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9df87b-b3f8-43d1-8692-bfeafa80e646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35bd47b95d534d7fbbcf744f79767565)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:43.990724 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9df87b-b3f8-43d1-8692-bfeafa80e646 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:44.042559 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1d9df87b-b3f8-43d1-8692-bfeafa80e646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35bd47b95d534d7fbbcf744f79767565', 'request_id': 'req-1d9df87b-b3f8-43d1-8692-bfeafa80e646'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.053027 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1d9df87b-b3f8-43d1-8692-bfeafa80e646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35bd47b95d534d7fbbcf744f79767565', 'request_id': 'req-1d9df87b-b3f8-43d1-8692-bfeafa80e646'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.059997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1d9df87b-b3f8-43d1-8692-bfeafa80e646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35bd47b95d534d7fbbcf744f79767565', 'request_id': 'req-1d9df87b-b3f8-43d1-8692-bfeafa80e646'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.071935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1d9df87b-b3f8-43d1-8692-bfeafa80e646 admin admin] The token cache is being invalidate because project 35bd47b95d534d7fbbcf744f79767565 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:44.072349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1d9df87b-b3f8-43d1-8692-bfeafa80e646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.073025 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1d9df87b-b3f8-43d1-8692-bfeafa80e646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.074597 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1356/2719] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:43 2022] DELETE /identity/v3/projects/35bd47b95d534d7fbbcf744f79767565 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:44.078858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bf8eb-314b-428d-a6fc-826b5454e1ea service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:44.089428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641bf8eb-314b-428d-a6fc-826b5454e1ea service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:44.094078 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:44.149273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eac6fc-b02b-4d0d-92d3-fc21348e973f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:44.149912 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eac6fc-b02b-4d0d-92d3-fc21348e973f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:44.150181 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eac6fc-b02b-4d0d-92d3-fc21348e973f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:44.150449 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eac6fc-b02b-4d0d-92d3-fc21348e973f admin admin] PATH_INFO: `/v3/users/564c8ea9c30c4ebf8f2e7cc8b3a13785` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:44.151641 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eac6fc-b02b-4d0d-92d3-fc21348e973f admin admin] RBAC: Authorizing `identity:delete_user(user_id=564c8ea9c30c4ebf8f2e7cc8b3a13785)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:44.153637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eac6fc-b02b-4d0d-92d3-fc21348e973f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:44.210050 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b9eac6fc-b02b-4d0d-92d3-fc21348e973f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '564c8ea9c30c4ebf8f2e7cc8b3a13785'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.219765 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b9eac6fc-b02b-4d0d-92d3-fc21348e973f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '564c8ea9c30c4ebf8f2e7cc8b3a13785'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.220617 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b9eac6fc-b02b-4d0d-92d3-fc21348e973f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '564c8ea9c30c4ebf8f2e7cc8b3a13785'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.221409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b9eac6fc-b02b-4d0d-92d3-fc21348e973f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '564c8ea9c30c4ebf8f2e7cc8b3a13785'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.234707 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-b9eac6fc-b02b-4d0d-92d3-fc21348e973f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '564c8ea9c30c4ebf8f2e7cc8b3a13785'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.247831 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1364/2720] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:44 2022] DELETE /identity/v3/users/564c8ea9c30c4ebf8f2e7cc8b3a13785 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:44.540936 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9df87b-b3f8-43d1-8692-bfeafa80e646 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:44.542495 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1d9df87b-b3f8-43d1-8692-bfeafa80e646 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:44.555941 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9df87b-b3f8-43d1-8692-bfeafa80e646 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:44.616039 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16755b57-174d-43c1-92b7-5d678c7b9aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:44.616613 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16755b57-174d-43c1-92b7-5d678c7b9aa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:44.616908 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16755b57-174d-43c1-92b7-5d678c7b9aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:44.617178 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16755b57-174d-43c1-92b7-5d678c7b9aa2 admin admin] PATH_INFO: `/v3/projects/e8eb992df4d041ae9a4e44b5a8504503` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:44.618289 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16755b57-174d-43c1-92b7-5d678c7b9aa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8eb992df4d041ae9a4e44b5a8504503)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:44.620233 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16755b57-174d-43c1-92b7-5d678c7b9aa2 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:44.655988 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:44.671038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-16755b57-174d-43c1-92b7-5d678c7b9aa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8eb992df4d041ae9a4e44b5a8504503', 'request_id': 'req-16755b57-174d-43c1-92b7-5d678c7b9aa2'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.681933 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-16755b57-174d-43c1-92b7-5d678c7b9aa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8eb992df4d041ae9a4e44b5a8504503', 'request_id': 'req-16755b57-174d-43c1-92b7-5d678c7b9aa2'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.688379 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-16755b57-174d-43c1-92b7-5d678c7b9aa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8eb992df4d041ae9a4e44b5a8504503', 'request_id': 'req-16755b57-174d-43c1-92b7-5d678c7b9aa2'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.700563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-16755b57-174d-43c1-92b7-5d678c7b9aa2 admin admin] The token cache is being invalidate because project e8eb992df4d041ae9a4e44b5a8504503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:44.700912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-16755b57-174d-43c1-92b7-5d678c7b9aa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.701639 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-16755b57-174d-43c1-92b7-5d678c7b9aa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.703162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1357/2721] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:44 2022] DELETE /identity/v3/projects/e8eb992df4d041ae9a4e44b5a8504503 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:44.708352 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4744450-566e-4ee4-add1-467d0d9352ee None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:44.708725 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4744450-566e-4ee4-add1-467d0d9352ee None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:44.709031 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4744450-566e-4ee4-add1-467d0d9352ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:44.732644 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f4744450-566e-4ee4-add1-467d0d9352ee None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:44.733993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f4744450-566e-4ee4-add1-467d0d9352ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:44.735045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16755b57-174d-43c1-92b7-5d678c7b9aa2 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:44.736549 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-16755b57-174d-43c1-92b7-5d678c7b9aa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:44.796832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1365/2722] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:44.804726 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16755b57-174d-43c1-92b7-5d678c7b9aa2 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:44.805835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf982f-9123-4f8e-9ee8-414aaef513b1 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:44.806099 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf982f-9123-4f8e-9ee8-414aaef513b1 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:44.806395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf982f-9123-4f8e-9ee8-414aaef513b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:44.810144 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081641dd-3a37-4512-abb8-488a2866db96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:44.810781 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081641dd-3a37-4512-abb8-488a2866db96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:44.811060 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081641dd-3a37-4512-abb8-488a2866db96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:44.811326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081641dd-3a37-4512-abb8-488a2866db96 admin admin] PATH_INFO: `/v3/users/28212643f5c14ded9663ef33316965d6` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:44.812509 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081641dd-3a37-4512-abb8-488a2866db96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28212643f5c14ded9663ef33316965d6)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:44.814415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081641dd-3a37-4512-abb8-488a2866db96 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:44.826225 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-17bf982f-9123-4f8e-9ee8-414aaef513b1 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:44.827431 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-17bf982f-9123-4f8e-9ee8-414aaef513b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:44.835360 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1366/2723] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:44.839288 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf982f-9123-4f8e-9ee8-414aaef513b1 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:44.848383 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf982f-9123-4f8e-9ee8-414aaef513b1 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:44.853067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfbdec-f31e-4047-b6dc-234e71d8ca97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:44.853648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfbdec-f31e-4047-b6dc-234e71d8ca97 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:44.853917 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfbdec-f31e-4047-b6dc-234e71d8ca97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:44.854183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfbdec-f31e-4047-b6dc-234e71d8ca97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:44.854812 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfbdec-f31e-4047-b6dc-234e71d8ca97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:15:44.855120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfbdec-f31e-4047-b6dc-234e71d8ca97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:44.856751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cfbdec-f31e-4047-b6dc-234e71d8ca97 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:44.863836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-081641dd-3a37-4512-abb8-488a2866db96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28212643f5c14ded9663ef33316965d6'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.868660 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1367/2724] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:15:44.872766 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfbdec-f31e-4047-b6dc-234e71d8ca97 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:44.874494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-081641dd-3a37-4512-abb8-488a2866db96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28212643f5c14ded9663ef33316965d6'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.875249 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-081641dd-3a37-4512-abb8-488a2866db96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28212643f5c14ded9663ef33316965d6'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.875906 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-081641dd-3a37-4512-abb8-488a2866db96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28212643f5c14ded9663ef33316965d6'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.883794 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cfbdec-f31e-4047-b6dc-234e71d8ca97 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:44.883956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-081641dd-3a37-4512-abb8-488a2866db96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28212643f5c14ded9663ef33316965d6'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:44.889192 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fc9d2-0548-4150-8a68-98f026aac31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:44.889784 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fc9d2-0548-4150-8a68-98f026aac31c admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:44.890057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fc9d2-0548-4150-8a68-98f026aac31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:44.890321 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558fc9d2-0548-4150-8a68-98f026aac31c admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:44.891019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fc9d2-0548-4150-8a68-98f026aac31c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:44.892933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558fc9d2-0548-4150-8a68-98f026aac31c admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:44.898640 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:44.906120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1358/2725] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:44 2022] DELETE /identity/v3/users/28212643f5c14ded9663ef33316965d6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:44.911021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa679c-78dc-4a67-90bf-71e396f1f4af None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:44.911287 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa679c-78dc-4a67-90bf-71e396f1f4af None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:44.911622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aa679c-78dc-4a67-90bf-71e396f1f4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:44.925997 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1368/2726] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:44 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:44.930194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fc9d2-0548-4150-8a68-98f026aac31c admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:44.932039 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-57aa679c-78dc-4a67-90bf-71e396f1f4af None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:44.932167 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-558fc9d2-0548-4150-8a68-98f026aac31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:44.933206 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-57aa679c-78dc-4a67-90bf-71e396f1f4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:44.966882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:44.996609 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558fc9d2-0548-4150-8a68-98f026aac31c admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:44.996922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1359/2727] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:45.001338 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e019a2d-5469-4c60-87ac-fe638d2e9938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.001950 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e019a2d-5469-4c60-87ac-fe638d2e9938 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.002221 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e019a2d-5469-4c60-87ac-fe638d2e9938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.002485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e019a2d-5469-4c60-87ac-fe638d2e9938 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.003109 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e019a2d-5469-4c60-87ac-fe638d2e9938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.005067 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e019a2d-5469-4c60-87ac-fe638d2e9938 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.007508 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa679c-78dc-4a67-90bf-71e396f1f4af None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.008967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-57aa679c-78dc-4a67-90bf-71e396f1f4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:45.030401 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1369/2728] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:44 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:45.031703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e019a2d-5469-4c60-87ac-fe638d2e9938 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.041415 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e019a2d-5469-4c60-87ac-fe638d2e9938 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.046270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea036e-3c3b-4f07-ad4e-b244b4dcce5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.047057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea036e-3c3b-4f07-ad4e-b244b4dcce5e admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.047395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea036e-3c3b-4f07-ad4e-b244b4dcce5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.047754 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea036e-3c3b-4f07-ad4e-b244b4dcce5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.048482 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea036e-3c3b-4f07-ad4e-b244b4dcce5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.050136 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea036e-3c3b-4f07-ad4e-b244b4dcce5e admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.061631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1370/2729] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:45.065557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea036e-3c3b-4f07-ad4e-b244b4dcce5e admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.066229 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aa679c-78dc-4a67-90bf-71e396f1f4af None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.069811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b10ab-2508-4de2-87a3-bc3254f635df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.070414 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b10ab-2508-4de2-87a3-bc3254f635df service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.070685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b10ab-2508-4de2-87a3-bc3254f635df service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.070948 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b10ab-2508-4de2-87a3-bc3254f635df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.075224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea036e-3c3b-4f07-ad4e-b244b4dcce5e admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.080068 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f78182-3158-4ec2-84b2-49f692d38e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.080668 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f78182-3158-4ec2-84b2-49f692d38e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.080937 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f78182-3158-4ec2-84b2-49f692d38e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.081202 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f78182-3158-4ec2-84b2-49f692d38e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.081442 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b10ab-2508-4de2-87a3-bc3254f635df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.081819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f78182-3158-4ec2-84b2-49f692d38e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.082891 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b10ab-2508-4de2-87a3-bc3254f635df service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.083402 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f78182-3158-4ec2-84b2-49f692d38e4f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.092400 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1360/2730] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:15:45.094066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1371/2731] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:45.098194 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b10ab-2508-4de2-87a3-bc3254f635df service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.108159 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b10ab-2508-4de2-87a3-bc3254f635df service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.113184 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a935b-e685-4868-aace-fcc282886376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.113816 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a935b-e685-4868-aace-fcc282886376 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.114094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a935b-e685-4868-aace-fcc282886376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.114392 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a935b-e685-4868-aace-fcc282886376 admin admin] PATH_INFO: `/v3/projects/672d31ea6a094d38ab1f9064b157d2cc/users/35797bdee23441ddbc4ebcb0b694f598/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.130079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a935b-e685-4868-aace-fcc282886376 admin admin] RBAC: Authorizing `identity:create_grant(project_id=672d31ea6a094d38ab1f9064b157d2cc, user_id=35797bdee23441ddbc4ebcb0b694f598, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.133043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a935b-e685-4868-aace-fcc282886376 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.146981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1361/2732] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:45 2022] PUT /identity/v3/projects/672d31ea6a094d38ab1f9064b157d2cc/users/35797bdee23441ddbc4ebcb0b694f598/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:45.157745 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e2e37-214a-48f4-909a-b341b83840cf None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.158073 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e2e37-214a-48f4-909a-b341b83840cf None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.158315 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e2e37-214a-48f4-909a-b341b83840cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.193766 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-bf4e2e37-214a-48f4-909a-b341b83840cf None None] MFA Rules not processed for user `35797bdee23441ddbc4ebcb0b694f598`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:45.194992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-bf4e2e37-214a-48f4-909a-b341b83840cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:45.274989 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1372/2733] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:45.279698 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a935b-e685-4868-aace-fcc282886376 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.358881 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a935b-e685-4868-aace-fcc282886376 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.364226 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66db9-8ea8-4edd-9e0b-aad0eb47ffad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.364848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66db9-8ea8-4edd-9e0b-aad0eb47ffad admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.365127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66db9-8ea8-4edd-9e0b-aad0eb47ffad admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.365398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b66db9-8ea8-4edd-9e0b-aad0eb47ffad admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.366053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b66db9-8ea8-4edd-9e0b-aad0eb47ffad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.368180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b66db9-8ea8-4edd-9e0b-aad0eb47ffad admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.398713 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1362/2734] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:45 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:45.403202 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e2e37-214a-48f4-909a-b341b83840cf None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.409049 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66db9-8ea8-4edd-9e0b-aad0eb47ffad admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.411326 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-80b66db9-8ea8-4edd-9e0b-aad0eb47ffad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:45.418162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4e2e37-214a-48f4-909a-b341b83840cf None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.477718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:45.479174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:45.496923 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5cd9f3-62bc-4284-ac04-4750a2005b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.497647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cd9f3-62bc-4284-ac04-4750a2005b23 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.497915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cd9f3-62bc-4284-ac04-4750a2005b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.498181 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cd9f3-62bc-4284-ac04-4750a2005b23 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.498915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5cd9f3-62bc-4284-ac04-4750a2005b23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.501290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5cd9f3-62bc-4284-ac04-4750a2005b23 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.507088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b66db9-8ea8-4edd-9e0b-aad0eb47ffad admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.512596 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dff72-0a4b-447d-a9dc-1178f1207c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.513660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dff72-0a4b-447d-a9dc-1178f1207c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.513660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dff72-0a4b-447d-a9dc-1178f1207c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.513909 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dff72-0a4b-447d-a9dc-1178f1207c09 admin admin] PATH_INFO: `/v3/projects/29210c432ea84bb0ae7135244d5b07e5` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.515079 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dff72-0a4b-447d-a9dc-1178f1207c09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29210c432ea84bb0ae7135244d5b07e5)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.517013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dff72-0a4b-447d-a9dc-1178f1207c09 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.524015 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1373/2735] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:45 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:45.528075 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5cd9f3-62bc-4284-ac04-4750a2005b23 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.537812 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5cd9f3-62bc-4284-ac04-4750a2005b23 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.542634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce3c04-0a0d-44f4-8728-b1eb33ce1da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.543338 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce3c04-0a0d-44f4-8728-b1eb33ce1da4 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.543763 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce3c04-0a0d-44f4-8728-b1eb33ce1da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.544199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ce3c04-0a0d-44f4-8728-b1eb33ce1da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.544864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce3c04-0a0d-44f4-8728-b1eb33ce1da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.546708 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ce3c04-0a0d-44f4-8728-b1eb33ce1da4 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.557785 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1374/2736] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:45.562175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce3c04-0a0d-44f4-8728-b1eb33ce1da4 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.568657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-295dff72-0a4b-447d-a9dc-1178f1207c09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29210c432ea84bb0ae7135244d5b07e5', 'request_id': 'req-295dff72-0a4b-447d-a9dc-1178f1207c09'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:45.572483 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ce3c04-0a0d-44f4-8728-b1eb33ce1da4 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.577535 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-295dff72-0a4b-447d-a9dc-1178f1207c09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29210c432ea84bb0ae7135244d5b07e5', 'request_id': 'req-295dff72-0a4b-447d-a9dc-1178f1207c09'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:45.583304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-295dff72-0a4b-447d-a9dc-1178f1207c09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29210c432ea84bb0ae7135244d5b07e5', 'request_id': 'req-295dff72-0a4b-447d-a9dc-1178f1207c09'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:45.595163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-295dff72-0a4b-447d-a9dc-1178f1207c09 admin admin] The token cache is being invalidate because project 29210c432ea84bb0ae7135244d5b07e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:45.595472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-295dff72-0a4b-447d-a9dc-1178f1207c09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:45.596143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-295dff72-0a4b-447d-a9dc-1178f1207c09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:45.597674 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1363/2737] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:45 2022] DELETE /identity/v3/projects/29210c432ea84bb0ae7135244d5b07e5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:45.602582 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8284c-8871-44fc-8f5c-daf515bce3c3 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.602898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8284c-8871-44fc-8f5c-daf515bce3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.603173 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8284c-8871-44fc-8f5c-daf515bce3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.623714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-c3c8284c-8871-44fc-8f5c-daf515bce3c3 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:45.624970 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c3c8284c-8871-44fc-8f5c-daf515bce3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:45.629773 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41efb55-42ad-4242-a2eb-168019ffdf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.630385 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41efb55-42ad-4242-a2eb-168019ffdf2c admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.630661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41efb55-42ad-4242-a2eb-168019ffdf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.630932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41efb55-42ad-4242-a2eb-168019ffdf2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.631639 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41efb55-42ad-4242-a2eb-168019ffdf2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.633262 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1364/2738] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:45.633408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41efb55-42ad-4242-a2eb-168019ffdf2c admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.641081 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f379dba-71c0-4583-8254-72156d2c1688 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.641295 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f379dba-71c0-4583-8254-72156d2c1688 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.641563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f379dba-71c0-4583-8254-72156d2c1688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.643119 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1375/2739] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:45.646959 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41efb55-42ad-4242-a2eb-168019ffdf2c admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.648618 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f41efb55-42ad-4242-a2eb-168019ffdf2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:45.660899 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41efb55-42ad-4242-a2eb-168019ffdf2c admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.661061 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-3f379dba-71c0-4583-8254-72156d2c1688 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:45.662181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3f379dba-71c0-4583-8254-72156d2c1688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:45.666158 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ed1b9-be0e-488c-8ea2-12591057eb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.666840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ed1b9-be0e-488c-8ea2-12591057eb42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.667114 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ed1b9-be0e-488c-8ea2-12591057eb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.667379 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ed1b9-be0e-488c-8ea2-12591057eb42 admin admin] PATH_INFO: `/v3/projects/6162b281dea048c8b7c4c956c17c0c22/users/ac1cc0194589414cb903ad77c12cc43a/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.670824 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1365/2740] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:45.674794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f379dba-71c0-4583-8254-72156d2c1688 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.682781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ed1b9-be0e-488c-8ea2-12591057eb42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6162b281dea048c8b7c4c956c17c0c22, user_id=ac1cc0194589414cb903ad77c12cc43a, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.683640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f379dba-71c0-4583-8254-72156d2c1688 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.685977 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ed1b9-be0e-488c-8ea2-12591057eb42 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.688776 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f2df8b-a040-46d1-914c-7af9323a2868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.689367 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2df8b-a040-46d1-914c-7af9323a2868 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.689643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2df8b-a040-46d1-914c-7af9323a2868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.689921 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f2df8b-a040-46d1-914c-7af9323a2868 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.690563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2df8b-a040-46d1-914c-7af9323a2868 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:15:45.690947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2df8b-a040-46d1-914c-7af9323a2868 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.692722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f2df8b-a040-46d1-914c-7af9323a2868 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.702199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1376/2741] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:45 2022] PUT /identity/v3/projects/6162b281dea048c8b7c4c956c17c0c22/users/ac1cc0194589414cb903ad77c12cc43a/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:45.713119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1366/2742] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:15:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:15:45.713260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ed1b9-be0e-488c-8ea2-12591057eb42 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.714598 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e46fc-11af-4760-9a91-43d7998d4997 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.715053 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e46fc-11af-4760-9a91-43d7998d4997 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.715241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e46fc-11af-4760-9a91-43d7998d4997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.743201 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-838e46fc-11af-4760-9a91-43d7998d4997 None None] MFA Rules not processed for user `ac1cc0194589414cb903ad77c12cc43a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:45.744438 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-838e46fc-11af-4760-9a91-43d7998d4997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:45.775806 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ed1b9-be0e-488c-8ea2-12591057eb42 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.781272 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829666a2-9201-4a35-88a9-bebd4dff8788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.781879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829666a2-9201-4a35-88a9-bebd4dff8788 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.782150 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829666a2-9201-4a35-88a9-bebd4dff8788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.782416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829666a2-9201-4a35-88a9-bebd4dff8788 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.783046 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829666a2-9201-4a35-88a9-bebd4dff8788 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.784987 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829666a2-9201-4a35-88a9-bebd4dff8788 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.809754 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1367/2743] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:45.812270 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1377/2744] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:45 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:45.816860 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e46fc-11af-4760-9a91-43d7998d4997 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.817485 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829666a2-9201-4a35-88a9-bebd4dff8788 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.827305 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e46fc-11af-4760-9a91-43d7998d4997 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.839496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:45.859187 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:45.882389 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829666a2-9201-4a35-88a9-bebd4dff8788 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.885266 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e818d-af0e-4226-bf9c-c34370afca7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.885995 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e818d-af0e-4226-bf9c-c34370afca7a admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.886273 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e818d-af0e-4226-bf9c-c34370afca7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.886545 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e818d-af0e-4226-bf9c-c34370afca7a admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.887034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d75449-eb50-46cf-9529-59c25d02db9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.887212 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e818d-af0e-4226-bf9c-c34370afca7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.887671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d75449-eb50-46cf-9529-59c25d02db9f admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.887949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d75449-eb50-46cf-9529-59c25d02db9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.888327 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d75449-eb50-46cf-9529-59c25d02db9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.888957 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d75449-eb50-46cf-9529-59c25d02db9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.889274 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e818d-af0e-4226-bf9c-c34370afca7a admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.890843 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d75449-eb50-46cf-9529-59c25d02db9f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.913531 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1368/2745] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:45 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:45.917405 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e818d-af0e-4226-bf9c-c34370afca7a admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.919233 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1378/2746] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:45 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:45.922866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d75449-eb50-46cf-9529-59c25d02db9f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:45.927669 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e818d-af0e-4226-bf9c-c34370afca7a admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.984230 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64e217-efe8-4b81-81ca-0a69dab03942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.984851 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64e217-efe8-4b81-81ca-0a69dab03942 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.985131 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64e217-efe8-4b81-81ca-0a69dab03942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.985402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64e217-efe8-4b81-81ca-0a69dab03942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.985507 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d75449-eb50-46cf-9529-59c25d02db9f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:45.986062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64e217-efe8-4b81-81ca-0a69dab03942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.987730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64e217-efe8-4b81-81ca-0a69dab03942 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.990162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6fece9-7422-4e8b-b772-431bd9fd4aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:45.990759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fece9-7422-4e8b-b772-431bd9fd4aad admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:45.991034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fece9-7422-4e8b-b772-431bd9fd4aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:45.991303 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fece9-7422-4e8b-b772-431bd9fd4aad admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:45.991968 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6fece9-7422-4e8b-b772-431bd9fd4aad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:45.993874 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6fece9-7422-4e8b-b772-431bd9fd4aad admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:45.998110 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1369/2747] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:46.002196 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64e217-efe8-4b81-81ca-0a69dab03942 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:46.012565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64e217-efe8-4b81-81ca-0a69dab03942 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:46.016924 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1379/2748] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:45 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:46.020107 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d22f8c-2832-4055-9bd0-6466d76d2c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:46.020707 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6fece9-7422-4e8b-b772-431bd9fd4aad admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:46.020814 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d22f8c-2832-4055-9bd0-6466d76d2c87 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.021014 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d22f8c-2832-4055-9bd0-6466d76d2c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.021563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d22f8c-2832-4055-9bd0-6466d76d2c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.022235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d22f8c-2832-4055-9bd0-6466d76d2c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:46.023901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d22f8c-2832-4055-9bd0-6466d76d2c87 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:46.030169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6fece9-7422-4e8b-b772-431bd9fd4aad admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:46.034673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0673d-d236-440f-97dd-bc9e4e19a332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:46.037993 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1370/2749] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:46.038125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0673d-d236-440f-97dd-bc9e4e19a332 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.038125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0673d-d236-440f-97dd-bc9e4e19a332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.038125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0673d-d236-440f-97dd-bc9e4e19a332 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.038125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0673d-d236-440f-97dd-bc9e4e19a332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:46.038125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0673d-d236-440f-97dd-bc9e4e19a332 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:46.039235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d22f8c-2832-4055-9bd0-6466d76d2c87 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:46.048648 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1380/2750] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:46.049696 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d22f8c-2832-4055-9bd0-6466d76d2c87 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:46.052494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0673d-d236-440f-97dd-bc9e4e19a332 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:46.054296 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4cfe69-2ed0-49f9-9bb0-a2a1c8c995d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:46.054938 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cfe69-2ed0-49f9-9bb0-a2a1c8c995d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.055214 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cfe69-2ed0-49f9-9bb0-a2a1c8c995d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.055518 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cfe69-2ed0-49f9-9bb0-a2a1c8c995d2 admin admin] PATH_INFO: `/v3/projects/15dca1e918ed4890909549ba1fb6946a/users/93f648d1a67044dbaa901900e6e0776a/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.063016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0673d-d236-440f-97dd-bc9e4e19a332 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:46.067835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f1b52-682b-4399-9dea-131757570b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:46.068467 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f1b52-682b-4399-9dea-131757570b53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.068740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f1b52-682b-4399-9dea-131757570b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.069005 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f1b52-682b-4399-9dea-131757570b53 admin admin] PATH_INFO: `/v3/projects/58490dcd2c6e44b1bac8a90b8f646934/users/69d5326c393343ef96c23a52f8c40f5a/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.070612 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4cfe69-2ed0-49f9-9bb0-a2a1c8c995d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15dca1e918ed4890909549ba1fb6946a, user_id=93f648d1a67044dbaa901900e6e0776a, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:46.073363 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4cfe69-2ed0-49f9-9bb0-a2a1c8c995d2 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:46.083850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f1b52-682b-4399-9dea-131757570b53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58490dcd2c6e44b1bac8a90b8f646934, user_id=69d5326c393343ef96c23a52f8c40f5a, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:46.086479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f1b52-682b-4399-9dea-131757570b53 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:46.086621 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1371/2751] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:46 2022] PUT /identity/v3/projects/15dca1e918ed4890909549ba1fb6946a/users/93f648d1a67044dbaa901900e6e0776a/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:46.098839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfa5c1-1000-4b28-bc82-806f5332afda None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.099117 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfa5c1-1000-4b28-bc82-806f5332afda None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.099383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfa5c1-1000-4b28-bc82-806f5332afda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.104180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1381/2752] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:46 2022] PUT /identity/v3/projects/58490dcd2c6e44b1bac8a90b8f646934/users/69d5326c393343ef96c23a52f8c40f5a/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:46.108102 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f1b52-682b-4399-9dea-131757570b53 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:46.127427 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-78bfa5c1-1000-4b28-bc82-806f5332afda None None] MFA Rules not processed for user `93f648d1a67044dbaa901900e6e0776a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:46.128661 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-78bfa5c1-1000-4b28-bc82-806f5332afda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:46.170944 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f1b52-682b-4399-9dea-131757570b53 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:46.175861 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430d3b1-46a8-4f16-8bcb-c27a5673d524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:46.176463 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430d3b1-46a8-4f16-8bcb-c27a5673d524 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.176734 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430d3b1-46a8-4f16-8bcb-c27a5673d524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.176997 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430d3b1-46a8-4f16-8bcb-c27a5673d524 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.177630 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430d3b1-46a8-4f16-8bcb-c27a5673d524 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:46.183533 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430d3b1-46a8-4f16-8bcb-c27a5673d524 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:46.183533 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:46.191547 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1382/2753] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:46.195594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430d3b1-46a8-4f16-8bcb-c27a5673d524 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:46.198667 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1372/2754] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:46.203632 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfa5c1-1000-4b28-bc82-806f5332afda None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:46.206581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430d3b1-46a8-4f16-8bcb-c27a5673d524 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:46.211320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab5291-9d5f-4bfd-9e3a-817e4cfa2610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:46.211988 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab5291-9d5f-4bfd-9e3a-817e4cfa2610 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.212260 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab5291-9d5f-4bfd-9e3a-817e4cfa2610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.212543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab5291-9d5f-4bfd-9e3a-817e4cfa2610 admin admin] PATH_INFO: `/v3/projects/58490dcd2c6e44b1bac8a90b8f646934/users/69d5326c393343ef96c23a52f8c40f5a/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.214111 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfa5c1-1000-4b28-bc82-806f5332afda None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:46.214443 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab5291-9d5f-4bfd-9e3a-817e4cfa2610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58490dcd2c6e44b1bac8a90b8f646934, user_id=69d5326c393343ef96c23a52f8c40f5a, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:46.217173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab5291-9d5f-4bfd-9e3a-817e4cfa2610 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:46.218721 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010dc85a-0e17-4a57-b40e-010f6c6cf1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:46.219301 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dc85a-0e17-4a57-b40e-010f6c6cf1ca admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.219628 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dc85a-0e17-4a57-b40e-010f6c6cf1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.219904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010dc85a-0e17-4a57-b40e-010f6c6cf1ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.220528 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010dc85a-0e17-4a57-b40e-010f6c6cf1ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:46.222424 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010dc85a-0e17-4a57-b40e-010f6c6cf1ca admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:46.227798 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1383/2755] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:46 2022] PUT /identity/v3/projects/58490dcd2c6e44b1bac8a90b8f646934/users/69d5326c393343ef96c23a52f8c40f5a/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:46.237904 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:46.257389 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1373/2756] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:46 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:46.258616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab5291-9d5f-4bfd-9e3a-817e4cfa2610 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:46.298717 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b7a69-8100-4827-9921-7a8211f3db91 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.299009 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b7a69-8100-4827-9921-7a8211f3db91 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.299275 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b7a69-8100-4827-9921-7a8211f3db91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.327841 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-895b7a69-8100-4827-9921-7a8211f3db91 None None] MFA Rules not processed for user `69d5326c393343ef96c23a52f8c40f5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:46.328756 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab5291-9d5f-4bfd-9e3a-817e4cfa2610 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:46.329327 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-895b7a69-8100-4827-9921-7a8211f3db91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:46.386448 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1cd6c-379e-4bfd-93e8-5314cbf0237e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:46.387064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1cd6c-379e-4bfd-93e8-5314cbf0237e admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.387335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1cd6c-379e-4bfd-93e8-5314cbf0237e admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.387728 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb1cd6c-379e-4bfd-93e8-5314cbf0237e admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.388429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1cd6c-379e-4bfd-93e8-5314cbf0237e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:46.390516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb1cd6c-379e-4bfd-93e8-5314cbf0237e admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:46.398178 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1374/2757] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:46.410845 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b7a69-8100-4827-9921-7a8211f3db91 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:46.412321 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-895b7a69-8100-4827-9921-7a8211f3db91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:46.413967 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1384/2758] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:46 2022] POST /identity/v3/users => generated 290 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:46.417521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1cd6c-379e-4bfd-93e8-5314cbf0237e admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:46.427088 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb1cd6c-379e-4bfd-93e8-5314cbf0237e admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:46.431703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263dd37-126c-4c3f-aec3-e98f62630d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:46.432285 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263dd37-126c-4c3f-aec3-e98f62630d58 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.432557 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263dd37-126c-4c3f-aec3-e98f62630d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.432825 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263dd37-126c-4c3f-aec3-e98f62630d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.433450 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263dd37-126c-4c3f-aec3-e98f62630d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:46.435393 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263dd37-126c-4c3f-aec3-e98f62630d58 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:46.449004 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1385/2759] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:46.449992 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263dd37-126c-4c3f-aec3-e98f62630d58 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:46.459640 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263dd37-126c-4c3f-aec3-e98f62630d58 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:46.464074 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130ae9d-8b4e-40b6-a1a8-53d8172cc03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:46.464710 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130ae9d-8b4e-40b6-a1a8-53d8172cc03d admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.464991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130ae9d-8b4e-40b6-a1a8-53d8172cc03d admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.465254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130ae9d-8b4e-40b6-a1a8-53d8172cc03d admin admin] PATH_INFO: `/v3/projects/b74e0d850cb3444f9675bac322de0a51/users/45f9d14830bd44669ca2f76a0f37617e/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.468853 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b7a69-8100-4827-9921-7a8211f3db91 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:46.472413 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92aae60-806f-481e-8f11-34ad59c101da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:46.473013 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92aae60-806f-481e-8f11-34ad59c101da service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.473290 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92aae60-806f-481e-8f11-34ad59c101da service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.473555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92aae60-806f-481e-8f11-34ad59c101da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.475272 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f92aae60-806f-481e-8f11-34ad59c101da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:46.479866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130ae9d-8b4e-40b6-a1a8-53d8172cc03d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b74e0d850cb3444f9675bac322de0a51, user_id=45f9d14830bd44669ca2f76a0f37617e, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:46.482664 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130ae9d-8b4e-40b6-a1a8-53d8172cc03d admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:46.496603 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1386/2760] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:46 2022] PUT /identity/v3/projects/b74e0d850cb3444f9675bac322de0a51/users/45f9d14830bd44669ca2f76a0f37617e/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:46.500918 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130ae9d-8b4e-40b6-a1a8-53d8172cc03d admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:46.549307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92aae60-806f-481e-8f11-34ad59c101da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:46.550854 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92aae60-806f-481e-8f11-34ad59c101da service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:46.551640 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:46.561425 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1375/2761] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:15:46.570381 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130ae9d-8b4e-40b6-a1a8-53d8172cc03d admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:46.575187 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604959e8-4fa5-4363-9ddf-89d4fe40e66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:46.575876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604959e8-4fa5-4363-9ddf-89d4fe40e66f admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.576157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604959e8-4fa5-4363-9ddf-89d4fe40e66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.576448 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604959e8-4fa5-4363-9ddf-89d4fe40e66f admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.577101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604959e8-4fa5-4363-9ddf-89d4fe40e66f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:46.578773 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604959e8-4fa5-4363-9ddf-89d4fe40e66f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:46.589084 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1387/2762] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:15:46.593488 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92aae60-806f-481e-8f11-34ad59c101da service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:46.605038 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92aae60-806f-481e-8f11-34ad59c101da service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:46.609926 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea47def-5261-4f2d-aa49-f97414e588f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:46.610593 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea47def-5261-4f2d-aa49-f97414e588f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.610842 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea47def-5261-4f2d-aa49-f97414e588f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.611120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea47def-5261-4f2d-aa49-f97414e588f6 admin admin] PATH_INFO: `/v3/projects/b74e0d850cb3444f9675bac322de0a51/users/45f9d14830bd44669ca2f76a0f37617e/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.613129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea47def-5261-4f2d-aa49-f97414e588f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b74e0d850cb3444f9675bac322de0a51, user_id=45f9d14830bd44669ca2f76a0f37617e, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:46.615813 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea47def-5261-4f2d-aa49-f97414e588f6 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:46.626857 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1376/2763] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:46 2022] PUT /identity/v3/projects/b74e0d850cb3444f9675bac322de0a51/users/45f9d14830bd44669ca2f76a0f37617e/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:46.637358 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc17ed-4150-48b3-ad2a-3894713bc44a None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.637641 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc17ed-4150-48b3-ad2a-3894713bc44a None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.637941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdc17ed-4150-48b3-ad2a-3894713bc44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.665750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-4cdc17ed-4150-48b3-ad2a-3894713bc44a None None] MFA Rules not processed for user `45f9d14830bd44669ca2f76a0f37617e`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:46.666964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-4cdc17ed-4150-48b3-ad2a-3894713bc44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:46.741387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1388/2764] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:46.753657 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea47def-5261-4f2d-aa49-f97414e588f6 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:46.755421 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-cea47def-5261-4f2d-aa49-f97414e588f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:46.782268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:46.831906 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea47def-5261-4f2d-aa49-f97414e588f6 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:46.835708 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3067f2-72dc-4ae2-b2f6-3789c4565fd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:46.836352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3067f2-72dc-4ae2-b2f6-3789c4565fd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.836628 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3067f2-72dc-4ae2-b2f6-3789c4565fd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.836898 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3067f2-72dc-4ae2-b2f6-3789c4565fd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:46.853956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3067f2-72dc-4ae2-b2f6-3789c4565fd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:46.855415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3067f2-72dc-4ae2-b2f6-3789c4565fd6 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:46.866411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1377/2765] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:46.951465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc17ed-4150-48b3-ad2a-3894713bc44a None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:46.961116 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdc17ed-4150-48b3-ad2a-3894713bc44a None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:46.964810 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64554f-0f2f-42e9-8a9a-03bca0491379 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:46.965408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64554f-0f2f-42e9-8a9a-03bca0491379 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:46.965681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64554f-0f2f-42e9-8a9a-03bca0491379 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:46.965946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64554f-0f2f-42e9-8a9a-03bca0491379 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:47.025273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64554f-0f2f-42e9-8a9a-03bca0491379 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:47.026729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64554f-0f2f-42e9-8a9a-03bca0491379 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:47.035979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1389/2766] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:49.170253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3067f2-72dc-4ae2-b2f6-3789c4565fd6 service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:49.180227 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3067f2-72dc-4ae2-b2f6-3789c4565fd6 service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:49.185242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ea3fd-59f0-4205-a3e8-0aa1ff4d2aa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:49.186028 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ea3fd-59f0-4205-a3e8-0aa1ff4d2aa1 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:49.186305 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ea3fd-59f0-4205-a3e8-0aa1ff4d2aa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:49.186580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747ea3fd-59f0-4205-a3e8-0aa1ff4d2aa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:49.251426 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ea3fd-59f0-4205-a3e8-0aa1ff4d2aa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:49.252835 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747ea3fd-59f0-4205-a3e8-0aa1ff4d2aa1 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:49.262268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1378/2767] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:50.146043 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64554f-0f2f-42e9-8a9a-03bca0491379 service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:50.155668 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64554f-0f2f-42e9-8a9a-03bca0491379 service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:50.160427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f10ae-5d4a-416e-9d61-f7c0ef8f1d56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:50.161048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f10ae-5d4a-416e-9d61-f7c0ef8f1d56 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:50.161293 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f10ae-5d4a-416e-9d61-f7c0ef8f1d56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:50.161559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f10ae-5d4a-416e-9d61-f7c0ef8f1d56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:50.222377 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f10ae-5d4a-416e-9d61-f7c0ef8f1d56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:50.223842 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f10ae-5d4a-416e-9d61-f7c0ef8f1d56 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:50.227720 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:50.234243 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1390/2768] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:52.869664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ea3fd-59f0-4205-a3e8-0aa1ff4d2aa1 service cinder] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:52.984947 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747ea3fd-59f0-4205-a3e8-0aa1ff4d2aa1 service cinder] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:52.989901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd51f7-d09f-4a3f-b5c4-e9600e5f9e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:52.990528 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd51f7-d09f-4a3f-b5c4-e9600e5f9e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:52.990811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd51f7-d09f-4a3f-b5c4-e9600e5f9e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:52.991080 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd51f7-d09f-4a3f-b5c4-e9600e5f9e1e admin admin] PATH_INFO: `/v3/users/93f648d1a67044dbaa901900e6e0776a` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:52.992282 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd51f7-d09f-4a3f-b5c4-e9600e5f9e1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=93f648d1a67044dbaa901900e6e0776a)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:52.994204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd51f7-d09f-4a3f-b5c4-e9600e5f9e1e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:53.017929 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:53.050625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-41dd51f7-d09f-4a3f-b5c4-e9600e5f9e1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93f648d1a67044dbaa901900e6e0776a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:53.060349 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-41dd51f7-d09f-4a3f-b5c4-e9600e5f9e1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93f648d1a67044dbaa901900e6e0776a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:53.060974 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-41dd51f7-d09f-4a3f-b5c4-e9600e5f9e1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93f648d1a67044dbaa901900e6e0776a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:53.061625 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-41dd51f7-d09f-4a3f-b5c4-e9600e5f9e1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93f648d1a67044dbaa901900e6e0776a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:53.069222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-41dd51f7-d09f-4a3f-b5c4-e9600e5f9e1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93f648d1a67044dbaa901900e6e0776a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:53.087512 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1379/2769] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:52 2022] DELETE /identity/v3/users/93f648d1a67044dbaa901900e6e0776a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:53.092249 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdce78-f4d2-4bc8-9c3a-8ee0a9e6be52 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:53.092853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdce78-f4d2-4bc8-9c3a-8ee0a9e6be52 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:53.092853 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bdce78-f4d2-4bc8-9c3a-8ee0a9e6be52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:53.145493 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-88bdce78-f4d2-4bc8-9c3a-8ee0a9e6be52 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:53.146678 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-88bdce78-f4d2-4bc8-9c3a-8ee0a9e6be52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:53.206790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1391/2770] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:53.217561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd51f7-d09f-4a3f-b5c4-e9600e5f9e1e admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:53.219139 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-41dd51f7-d09f-4a3f-b5c4-e9600e5f9e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:53.295507 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd51f7-d09f-4a3f-b5c4-e9600e5f9e1e admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:53.299218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa89276-e9c2-46ec-89b0-95889f0a6f3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:53.299857 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa89276-e9c2-46ec-89b0-95889f0a6f3e service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:53.300138 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa89276-e9c2-46ec-89b0-95889f0a6f3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:53.300409 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa89276-e9c2-46ec-89b0-95889f0a6f3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:53.310937 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa89276-e9c2-46ec-89b0-95889f0a6f3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:53.312809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa89276-e9c2-46ec-89b0-95889f0a6f3e service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:53.324474 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1380/2771] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:53.606254 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdce78-f4d2-4bc8-9c3a-8ee0a9e6be52 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:53.607807 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-88bdce78-f4d2-4bc8-9c3a-8ee0a9e6be52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:53.622830 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bdce78-f4d2-4bc8-9c3a-8ee0a9e6be52 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:53.629761 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc72719-9f73-4bd3-9d22-02b2a29edc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:53.630497 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc72719-9f73-4bd3-9d22-02b2a29edc99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:53.630674 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc72719-9f73-4bd3-9d22-02b2a29edc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:53.630936 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc72719-9f73-4bd3-9d22-02b2a29edc99 admin admin] PATH_INFO: `/v3/projects/15dca1e918ed4890909549ba1fb6946a` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:53.632336 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc72719-9f73-4bd3-9d22-02b2a29edc99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15dca1e918ed4890909549ba1fb6946a)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:53.634282 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc72719-9f73-4bd3-9d22-02b2a29edc99 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:53.687256 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1bc72719-9f73-4bd3-9d22-02b2a29edc99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15dca1e918ed4890909549ba1fb6946a', 'request_id': 'req-1bc72719-9f73-4bd3-9d22-02b2a29edc99'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:53.696729 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1bc72719-9f73-4bd3-9d22-02b2a29edc99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15dca1e918ed4890909549ba1fb6946a', 'request_id': 'req-1bc72719-9f73-4bd3-9d22-02b2a29edc99'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:53.702805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1bc72719-9f73-4bd3-9d22-02b2a29edc99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15dca1e918ed4890909549ba1fb6946a', 'request_id': 'req-1bc72719-9f73-4bd3-9d22-02b2a29edc99'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:53.715133 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1bc72719-9f73-4bd3-9d22-02b2a29edc99 admin admin] The token cache is being invalidate because project 15dca1e918ed4890909549ba1fb6946a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:53.715465 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1bc72719-9f73-4bd3-9d22-02b2a29edc99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:53.716295 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1bc72719-9f73-4bd3-9d22-02b2a29edc99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:53.717934 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1392/2772] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:53 2022] DELETE /identity/v3/projects/15dca1e918ed4890909549ba1fb6946a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:53.722377 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa89276-e9c2-46ec-89b0-95889f0a6f3e service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:53.732570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa89276-e9c2-46ec-89b0-95889f0a6f3e service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:53.792685 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f488b6-dfac-42bf-b789-96f3adaba879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:53.793330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f488b6-dfac-42bf-b789-96f3adaba879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:53.793602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f488b6-dfac-42bf-b789-96f3adaba879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:53.793956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f488b6-dfac-42bf-b789-96f3adaba879 admin admin] PATH_INFO: `/v3/users/45f9d14830bd44669ca2f76a0f37617e` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:53.795143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f488b6-dfac-42bf-b789-96f3adaba879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45f9d14830bd44669ca2f76a0f37617e)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:53.797083 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f488b6-dfac-42bf-b789-96f3adaba879 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:53.833065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:53.852849 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b8f488b6-dfac-42bf-b789-96f3adaba879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45f9d14830bd44669ca2f76a0f37617e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:53.860532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b8f488b6-dfac-42bf-b789-96f3adaba879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45f9d14830bd44669ca2f76a0f37617e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:53.861248 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b8f488b6-dfac-42bf-b789-96f3adaba879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45f9d14830bd44669ca2f76a0f37617e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:53.861992 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b8f488b6-dfac-42bf-b789-96f3adaba879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45f9d14830bd44669ca2f76a0f37617e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:53.869253 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-b8f488b6-dfac-42bf-b789-96f3adaba879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45f9d14830bd44669ca2f76a0f37617e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:53.887842 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1381/2773] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:53 2022] DELETE /identity/v3/users/45f9d14830bd44669ca2f76a0f37617e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:54.178202 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc72719-9f73-4bd3-9d22-02b2a29edc99 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:54.179704 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-1bc72719-9f73-4bd3-9d22-02b2a29edc99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:54.193440 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc72719-9f73-4bd3-9d22-02b2a29edc99 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:54.245576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:54.253696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda069a3-1778-409d-9b52-83c7d17793c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:54.254410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda069a3-1778-409d-9b52-83c7d17793c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:54.254905 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda069a3-1778-409d-9b52-83c7d17793c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:54.255550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda069a3-1778-409d-9b52-83c7d17793c1 admin admin] PATH_INFO: `/v3/projects/b74e0d850cb3444f9675bac322de0a51` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:54.256960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda069a3-1778-409d-9b52-83c7d17793c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b74e0d850cb3444f9675bac322de0a51)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:54.259600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda069a3-1778-409d-9b52-83c7d17793c1 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:54.309997 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-fda069a3-1778-409d-9b52-83c7d17793c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b74e0d850cb3444f9675bac322de0a51', 'request_id': 'req-fda069a3-1778-409d-9b52-83c7d17793c1'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:54.319498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-fda069a3-1778-409d-9b52-83c7d17793c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b74e0d850cb3444f9675bac322de0a51', 'request_id': 'req-fda069a3-1778-409d-9b52-83c7d17793c1'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:54.327993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-fda069a3-1778-409d-9b52-83c7d17793c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b74e0d850cb3444f9675bac322de0a51', 'request_id': 'req-fda069a3-1778-409d-9b52-83c7d17793c1'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:54.339719 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-fda069a3-1778-409d-9b52-83c7d17793c1 admin admin] The token cache is being invalidate because project b74e0d850cb3444f9675bac322de0a51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:54.340010 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-fda069a3-1778-409d-9b52-83c7d17793c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:54.340659 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-fda069a3-1778-409d-9b52-83c7d17793c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:54.342169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1393/2774] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:54 2022] DELETE /identity/v3/projects/b74e0d850cb3444f9675bac322de0a51 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:54.347269 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7cde1-107c-4092-b230-bb92b8e29d86 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:54.347661 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7cde1-107c-4092-b230-bb92b8e29d86 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:54.348143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7cde1-107c-4092-b230-bb92b8e29d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:54.400190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-abc7cde1-107c-4092-b230-bb92b8e29d86 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:54.401401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-abc7cde1-107c-4092-b230-bb92b8e29d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:54.462920 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1382/2775] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:54.472637 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333bd17f-2fef-435b-aee6-139fefa0dec5 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:54.472913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333bd17f-2fef-435b-aee6-139fefa0dec5 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:54.473183 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333bd17f-2fef-435b-aee6-139fefa0dec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:54.494464 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-333bd17f-2fef-435b-aee6-139fefa0dec5 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:54.495730 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-333bd17f-2fef-435b-aee6-139fefa0dec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:54.504851 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1394/2776] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:54.509298 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7cde1-107c-4092-b230-bb92b8e29d86 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:54.519620 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7cde1-107c-4092-b230-bb92b8e29d86 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:54.524370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aaeb0e-e193-4199-a6aa-8b50c661a89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:54.524980 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aaeb0e-e193-4199-a6aa-8b50c661a89d admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:54.525257 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aaeb0e-e193-4199-a6aa-8b50c661a89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:54.525530 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aaeb0e-e193-4199-a6aa-8b50c661a89d admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:54.526181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aaeb0e-e193-4199-a6aa-8b50c661a89d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:15:54.526484 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aaeb0e-e193-4199-a6aa-8b50c661a89d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:54.528153 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aaeb0e-e193-4199-a6aa-8b50c661a89d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:54.540872 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1383/2777] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:15:54.546182 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333bd17f-2fef-435b-aee6-139fefa0dec5 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:54.555676 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333bd17f-2fef-435b-aee6-139fefa0dec5 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:54.561034 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a5927-4980-4f89-b169-6a5e24441c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:54.561639 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a5927-4980-4f89-b169-6a5e24441c20 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:54.561976 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a5927-4980-4f89-b169-6a5e24441c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:54.562267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a5927-4980-4f89-b169-6a5e24441c20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:54.562912 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a5927-4980-4f89-b169-6a5e24441c20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:54.564850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733a5927-4980-4f89-b169-6a5e24441c20 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:54.593521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1395/2778] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:54 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:54.598258 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aaeb0e-e193-4199-a6aa-8b50c661a89d admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:54.613283 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aaeb0e-e193-4199-a6aa-8b50c661a89d admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:54.674524 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fe749-f6ad-4558-b61f-4884d47d9011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:54.675123 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fe749-f6ad-4558-b61f-4884d47d9011 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:54.675398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fe749-f6ad-4558-b61f-4884d47d9011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:54.675692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6fe749-f6ad-4558-b61f-4884d47d9011 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:54.676325 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6fe749-f6ad-4558-b61f-4884d47d9011 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:54.678222 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6fe749-f6ad-4558-b61f-4884d47d9011 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:54.697420 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:54.703802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1384/2779] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:54 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:54.708153 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a5927-4980-4f89-b169-6a5e24441c20 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:54.718914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733a5927-4980-4f89-b169-6a5e24441c20 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:54.723784 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e737c59-faf3-4b28-ab3a-2e7675fc6de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:54.724412 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e737c59-faf3-4b28-ab3a-2e7675fc6de7 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:54.724694 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e737c59-faf3-4b28-ab3a-2e7675fc6de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:54.724940 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e737c59-faf3-4b28-ab3a-2e7675fc6de7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:54.725597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e737c59-faf3-4b28-ab3a-2e7675fc6de7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:54.727205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e737c59-faf3-4b28-ab3a-2e7675fc6de7 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:54.737148 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1396/2780] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:54.741265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fe749-f6ad-4558-b61f-4884d47d9011 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:54.751688 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6fe749-f6ad-4558-b61f-4884d47d9011 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:54.757206 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26773fb-c84c-43eb-889c-2292df49b453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:54.757822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26773fb-c84c-43eb-889c-2292df49b453 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:54.758089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26773fb-c84c-43eb-889c-2292df49b453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:54.758358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26773fb-c84c-43eb-889c-2292df49b453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:54.759006 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26773fb-c84c-43eb-889c-2292df49b453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:54.760698 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26773fb-c84c-43eb-889c-2292df49b453 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:54.772030 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1385/2781] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:54.776176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e737c59-faf3-4b28-ab3a-2e7675fc6de7 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:54.786595 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e737c59-faf3-4b28-ab3a-2e7675fc6de7 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:54.791689 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c00f0-37bc-4678-b81c-4a3036e085d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:54.792332 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c00f0-37bc-4678-b81c-4a3036e085d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:54.792609 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c00f0-37bc-4678-b81c-4a3036e085d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:54.792879 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c00f0-37bc-4678-b81c-4a3036e085d1 admin admin] PATH_INFO: `/v3/projects/71ffd779e1b748fb97e683855a571933/users/4003ae2411814f6dadc5c9af68369fe5/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:54.808155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c00f0-37bc-4678-b81c-4a3036e085d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71ffd779e1b748fb97e683855a571933, user_id=4003ae2411814f6dadc5c9af68369fe5, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:54.811101 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c00f0-37bc-4678-b81c-4a3036e085d1 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:54.826040 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1397/2782] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:54 2022] PUT /identity/v3/projects/71ffd779e1b748fb97e683855a571933/users/4003ae2411814f6dadc5c9af68369fe5/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:54.836539 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a877c-db9b-46c2-bb6e-73d1a3b22666 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:54.836824 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a877c-db9b-46c2-bb6e-73d1a3b22666 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:54.837098 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a877c-db9b-46c2-bb6e-73d1a3b22666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:54.865172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-166a877c-db9b-46c2-bb6e-73d1a3b22666 None None] MFA Rules not processed for user `4003ae2411814f6dadc5c9af68369fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:54.866391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-166a877c-db9b-46c2-bb6e-73d1a3b22666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:54.933368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1386/2783] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:54.945828 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c00f0-37bc-4678-b81c-4a3036e085d1 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:54.947944 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-a57c00f0-37bc-4678-b81c-4a3036e085d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:55.007561 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c00f0-37bc-4678-b81c-4a3036e085d1 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:55.012429 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0aa541-b1a6-46ea-8ba0-3d967cf0363a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:55.013035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0aa541-b1a6-46ea-8ba0-3d967cf0363a service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:55.013312 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0aa541-b1a6-46ea-8ba0-3d967cf0363a service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:55.013576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0aa541-b1a6-46ea-8ba0-3d967cf0363a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:55.023028 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0aa541-b1a6-46ea-8ba0-3d967cf0363a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:55.024518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0aa541-b1a6-46ea-8ba0-3d967cf0363a service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:55.033687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1398/2784] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:15:55.053708 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a877c-db9b-46c2-bb6e-73d1a3b22666 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:55.128183 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a877c-db9b-46c2-bb6e-73d1a3b22666 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:55.134043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e0924a-8fb4-4a17-86a7-efb91f81c2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:55.134679 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0924a-8fb4-4a17-86a7-efb91f81c2c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:55.134951 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0924a-8fb4-4a17-86a7-efb91f81c2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:55.135213 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e0924a-8fb4-4a17-86a7-efb91f81c2c5 admin admin] PATH_INFO: `/v3/users/4003ae2411814f6dadc5c9af68369fe5` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:55.136517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0924a-8fb4-4a17-86a7-efb91f81c2c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4003ae2411814f6dadc5c9af68369fe5)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:55.138519 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e0924a-8fb4-4a17-86a7-efb91f81c2c5 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:55.197279 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e4e0924a-8fb4-4a17-86a7-efb91f81c2c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4003ae2411814f6dadc5c9af68369fe5'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.208339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e4e0924a-8fb4-4a17-86a7-efb91f81c2c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4003ae2411814f6dadc5c9af68369fe5'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.208339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e4e0924a-8fb4-4a17-86a7-efb91f81c2c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4003ae2411814f6dadc5c9af68369fe5'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.208339 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e4e0924a-8fb4-4a17-86a7-efb91f81c2c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4003ae2411814f6dadc5c9af68369fe5'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.209545 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0aa541-b1a6-46ea-8ba0-3d967cf0363a service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:55.211145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ce0aa541-b1a6-46ea-8ba0-3d967cf0363a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:55.215366 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-e4e0924a-8fb4-4a17-86a7-efb91f81c2c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4003ae2411814f6dadc5c9af68369fe5'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.226317 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0aa541-b1a6-46ea-8ba0-3d967cf0363a service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:55.227728 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:55.231391 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a389f4-3bab-417d-b192-1fd0296433ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:55.232198 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a389f4-3bab-417d-b192-1fd0296433ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:55.232556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a389f4-3bab-417d-b192-1fd0296433ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:55.232935 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a389f4-3bab-417d-b192-1fd0296433ad admin admin] PATH_INFO: `/v3/users/35797bdee23441ddbc4ebcb0b694f598` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:55.234852 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a389f4-3bab-417d-b192-1fd0296433ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=35797bdee23441ddbc4ebcb0b694f598)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:55.238629 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1387/2785] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:55 2022] DELETE /identity/v3/users/4003ae2411814f6dadc5c9af68369fe5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:55.238802 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a389f4-3bab-417d-b192-1fd0296433ad admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:55.239960 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e293c-bad2-484d-9ba4-12b1e01b4df8 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:55.240238 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e293c-bad2-484d-9ba4-12b1e01b4df8 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:55.240506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e293c-bad2-484d-9ba4-12b1e01b4df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:55.259428 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:55.259622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-ed3e293c-bad2-484d-9ba4-12b1e01b4df8 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:55.260859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ed3e293c-bad2-484d-9ba4-12b1e01b4df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:55.288364 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d5a389f4-3bab-417d-b192-1fd0296433ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35797bdee23441ddbc4ebcb0b694f598'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.296858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d5a389f4-3bab-417d-b192-1fd0296433ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35797bdee23441ddbc4ebcb0b694f598'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.297565 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d5a389f4-3bab-417d-b192-1fd0296433ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35797bdee23441ddbc4ebcb0b694f598'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.298251 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d5a389f4-3bab-417d-b192-1fd0296433ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35797bdee23441ddbc4ebcb0b694f598'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.311218 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d5a389f4-3bab-417d-b192-1fd0296433ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35797bdee23441ddbc4ebcb0b694f598'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.321634 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1388/2786] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:55.325857 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1399/2787] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:55 2022] DELETE /identity/v3/users/35797bdee23441ddbc4ebcb0b694f598 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:55.329839 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d45f2-9ee9-45c3-b366-5a2f0b552c62 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:55.330127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d45f2-9ee9-45c3-b366-5a2f0b552c62 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:55.330396 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d45f2-9ee9-45c3-b366-5a2f0b552c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:55.332044 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a389f4-3bab-417d-b192-1fd0296433ad admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:55.333518 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-d5a389f4-3bab-417d-b192-1fd0296433ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:55.349767 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-227d45f2-9ee9-45c3-b366-5a2f0b552c62 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:55.350963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-227d45f2-9ee9-45c3-b366-5a2f0b552c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:55.391574 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a389f4-3bab-417d-b192-1fd0296433ad admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:55.395575 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee8c85-233e-457c-a2f5-d0fbc5cce23b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:55.396173 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee8c85-233e-457c-a2f5-d0fbc5cce23b service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:55.396446 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee8c85-233e-457c-a2f5-d0fbc5cce23b service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:55.396711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee8c85-233e-457c-a2f5-d0fbc5cce23b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:55.409535 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1389/2788] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:55.419506 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d45f2-9ee9-45c3-b366-5a2f0b552c62 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:55.429629 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d45f2-9ee9-45c3-b366-5a2f0b552c62 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:55.434285 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099548b-194a-420d-8303-19e397a2a879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:55.434860 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099548b-194a-420d-8303-19e397a2a879 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:55.435124 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099548b-194a-420d-8303-19e397a2a879 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:55.435389 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3099548b-194a-420d-8303-19e397a2a879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:55.445109 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3099548b-194a-420d-8303-19e397a2a879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:55.447103 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3099548b-194a-420d-8303-19e397a2a879 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:55.458149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1390/2789] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:15:55.459543 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee8c85-233e-457c-a2f5-d0fbc5cce23b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:55.460990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebee8c85-233e-457c-a2f5-d0fbc5cce23b service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:55.472375 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1400/2790] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:15:55.850809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099548b-194a-420d-8303-19e397a2a879 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:55.852294 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3099548b-194a-420d-8303-19e397a2a879 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:55.858850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee8c85-233e-457c-a2f5-d0fbc5cce23b service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:55.860542 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ebee8c85-233e-457c-a2f5-d0fbc5cce23b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:55.875128 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebee8c85-233e-457c-a2f5-d0fbc5cce23b service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:55.880494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac95198-10b6-45c2-8202-e7a7708f11cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:55.881144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac95198-10b6-45c2-8202-e7a7708f11cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:55.881531 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac95198-10b6-45c2-8202-e7a7708f11cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:55.881681 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac95198-10b6-45c2-8202-e7a7708f11cb admin admin] PATH_INFO: `/v3/projects/71ffd779e1b748fb97e683855a571933` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:55.882930 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac95198-10b6-45c2-8202-e7a7708f11cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=71ffd779e1b748fb97e683855a571933)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:55.883814 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3099548b-194a-420d-8303-19e397a2a879 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:55.884926 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac95198-10b6-45c2-8202-e7a7708f11cb admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:55.889489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5b235-f26f-4773-bc24-fd851bb6e893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:55.890184 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5b235-f26f-4773-bc24-fd851bb6e893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:55.890458 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5b235-f26f-4773-bc24-fd851bb6e893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:55.891293 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5b235-f26f-4773-bc24-fd851bb6e893 admin admin] PATH_INFO: `/v3/projects/672d31ea6a094d38ab1f9064b157d2cc` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:55.893085 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5b235-f26f-4773-bc24-fd851bb6e893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=672d31ea6a094d38ab1f9064b157d2cc)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:55.895724 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5b235-f26f-4773-bc24-fd851bb6e893 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:55.930243 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:55.938600 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-bac95198-10b6-45c2-8202-e7a7708f11cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71ffd779e1b748fb97e683855a571933', 'request_id': 'req-bac95198-10b6-45c2-8202-e7a7708f11cb'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.948115 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-bac95198-10b6-45c2-8202-e7a7708f11cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71ffd779e1b748fb97e683855a571933', 'request_id': 'req-bac95198-10b6-45c2-8202-e7a7708f11cb'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.949556 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-94f5b235-f26f-4773-bc24-fd851bb6e893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '672d31ea6a094d38ab1f9064b157d2cc', 'request_id': 'req-94f5b235-f26f-4773-bc24-fd851bb6e893'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.954905 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-bac95198-10b6-45c2-8202-e7a7708f11cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71ffd779e1b748fb97e683855a571933', 'request_id': 'req-bac95198-10b6-45c2-8202-e7a7708f11cb'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.957945 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-94f5b235-f26f-4773-bc24-fd851bb6e893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '672d31ea6a094d38ab1f9064b157d2cc', 'request_id': 'req-94f5b235-f26f-4773-bc24-fd851bb6e893'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.963881 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-94f5b235-f26f-4773-bc24-fd851bb6e893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '672d31ea6a094d38ab1f9064b157d2cc', 'request_id': 'req-94f5b235-f26f-4773-bc24-fd851bb6e893'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.967106 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-bac95198-10b6-45c2-8202-e7a7708f11cb admin admin] The token cache is being invalidate because project 71ffd779e1b748fb97e683855a571933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:55.967390 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-bac95198-10b6-45c2-8202-e7a7708f11cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.968118 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-bac95198-10b6-45c2-8202-e7a7708f11cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.969726 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1401/2791] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:55 2022] DELETE /identity/v3/projects/71ffd779e1b748fb97e683855a571933 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:55.974395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434a6a4-1a40-4ec7-9fb5-45e0b54ed2b7 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:55.974675 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434a6a4-1a40-4ec7-9fb5-45e0b54ed2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:55.974946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c434a6a4-1a40-4ec7-9fb5-45e0b54ed2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:55.975503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-94f5b235-f26f-4773-bc24-fd851bb6e893 admin admin] The token cache is being invalidate because project 672d31ea6a094d38ab1f9064b157d2cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:55.975798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-94f5b235-f26f-4773-bc24-fd851bb6e893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.976442 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-94f5b235-f26f-4773-bc24-fd851bb6e893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:55.977981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1391/2792] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:55 2022] DELETE /identity/v3/projects/672d31ea6a094d38ab1f9064b157d2cc => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:55.981718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5b235-f26f-4773-bc24-fd851bb6e893 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:55.983043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-94f5b235-f26f-4773-bc24-fd851bb6e893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:55.992835 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:55.994643 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-c434a6a4-1a40-4ec7-9fb5-45e0b54ed2b7 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:55.995954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c434a6a4-1a40-4ec7-9fb5-45e0b54ed2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:56.051755 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5b235-f26f-4773-bc24-fd851bb6e893 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:56.056568 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1402/2793] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:56.056733 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f0144-11bf-4e59-8136-78157ecf9866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:56.057241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f0144-11bf-4e59-8136-78157ecf9866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:56.057517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f0144-11bf-4e59-8136-78157ecf9866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:56.057787 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690f0144-11bf-4e59-8136-78157ecf9866 admin admin] PATH_INFO: `/v3/users/ac1cc0194589414cb903ad77c12cc43a` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:56.059267 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f0144-11bf-4e59-8136-78157ecf9866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1cc0194589414cb903ad77c12cc43a)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:56.061242 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690f0144-11bf-4e59-8136-78157ecf9866 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:56.064489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d85672-e64e-4d73-8dc1-6c721518ddc1 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:56.064777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d85672-e64e-4d73-8dc1-6c721518ddc1 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:56.065403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d85672-e64e-4d73-8dc1-6c721518ddc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:56.086407 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-63d85672-e64e-4d73-8dc1-6c721518ddc1 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:56.087841 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-63d85672-e64e-4d73-8dc1-6c721518ddc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:56.097519 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1403/2794] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:56.101726 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d85672-e64e-4d73-8dc1-6c721518ddc1 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:56.108207 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-690f0144-11bf-4e59-8136-78157ecf9866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1cc0194589414cb903ad77c12cc43a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:56.110556 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d85672-e64e-4d73-8dc1-6c721518ddc1 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:56.115567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a5146-1973-4e1f-a6f8-3e4a86adfbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:56.116255 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a5146-1973-4e1f-a6f8-3e4a86adfbfb admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:56.116451 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-690f0144-11bf-4e59-8136-78157ecf9866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1cc0194589414cb903ad77c12cc43a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:56.116619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a5146-1973-4e1f-a6f8-3e4a86adfbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:56.116870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a5146-1973-4e1f-a6f8-3e4a86adfbfb admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:56.117181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-690f0144-11bf-4e59-8136-78157ecf9866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1cc0194589414cb903ad77c12cc43a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:56.117559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a5146-1973-4e1f-a6f8-3e4a86adfbfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:15:56.117794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-690f0144-11bf-4e59-8136-78157ecf9866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1cc0194589414cb903ad77c12cc43a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:56.117906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a5146-1973-4e1f-a6f8-3e4a86adfbfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:56.119469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a5146-1973-4e1f-a6f8-3e4a86adfbfb admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:56.125919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-690f0144-11bf-4e59-8136-78157ecf9866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1cc0194589414cb903ad77c12cc43a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:56.133091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1404/2795] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:15:56.137670 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a5146-1973-4e1f-a6f8-3e4a86adfbfb admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:56.139348 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ed8a5146-1973-4e1f-a6f8-3e4a86adfbfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:56.144510 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1392/2796] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:55 2022] DELETE /identity/v3/users/ac1cc0194589414cb903ad77c12cc43a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:56.152472 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a5146-1973-4e1f-a6f8-3e4a86adfbfb admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:56.216149 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a161091-9fa8-425d-8b62-305719ac89a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:56.216838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a161091-9fa8-425d-8b62-305719ac89a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:56.217167 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a161091-9fa8-425d-8b62-305719ac89a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:56.217498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a161091-9fa8-425d-8b62-305719ac89a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:56.218250 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a161091-9fa8-425d-8b62-305719ac89a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:56.220262 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a161091-9fa8-425d-8b62-305719ac89a8 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:56.249828 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1405/2797] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:56 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:56.254154 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f0144-11bf-4e59-8136-78157ecf9866 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:56.264632 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690f0144-11bf-4e59-8136-78157ecf9866 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:56.322561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d42610-0e58-4e0d-90e9-50c0848998b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:56.323180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d42610-0e58-4e0d-90e9-50c0848998b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:56.323468 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d42610-0e58-4e0d-90e9-50c0848998b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:56.323751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d42610-0e58-4e0d-90e9-50c0848998b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:56.324384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d42610-0e58-4e0d-90e9-50c0848998b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:56.326283 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d42610-0e58-4e0d-90e9-50c0848998b2 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:56.350584 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1393/2798] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:56 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:15:56.354803 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a161091-9fa8-425d-8b62-305719ac89a8 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:56.365957 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a161091-9fa8-425d-8b62-305719ac89a8 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:56.371051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd529975-2bdf-44bc-82a0-8507ff789dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:56.371687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd529975-2bdf-44bc-82a0-8507ff789dfe admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:56.371960 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd529975-2bdf-44bc-82a0-8507ff789dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:56.372224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd529975-2bdf-44bc-82a0-8507ff789dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:56.372867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd529975-2bdf-44bc-82a0-8507ff789dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:56.374602 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd529975-2bdf-44bc-82a0-8507ff789dfe admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:56.385169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1406/2799] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:56.389316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d42610-0e58-4e0d-90e9-50c0848998b2 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:56.399812 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d42610-0e58-4e0d-90e9-50c0848998b2 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:56.405031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d0f32-1bfb-40dc-abfb-c8ed0aab437f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:56.405742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0f32-1bfb-40dc-abfb-c8ed0aab437f admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:56.406052 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0f32-1bfb-40dc-abfb-c8ed0aab437f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:56.406334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d0f32-1bfb-40dc-abfb-c8ed0aab437f admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:56.406987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d0f32-1bfb-40dc-abfb-c8ed0aab437f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:56.408666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d0f32-1bfb-40dc-abfb-c8ed0aab437f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:56.419132 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1394/2800] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:56.423909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd529975-2bdf-44bc-82a0-8507ff789dfe admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:56.435509 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd529975-2bdf-44bc-82a0-8507ff789dfe admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:56.441053 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41183326-5b1e-440e-a6e6-144e0198937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:56.441913 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41183326-5b1e-440e-a6e6-144e0198937c admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:56.442395 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41183326-5b1e-440e-a6e6-144e0198937c admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:56.442564 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d0f32-1bfb-40dc-abfb-c8ed0aab437f admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:56.442707 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41183326-5b1e-440e-a6e6-144e0198937c admin admin] PATH_INFO: `/v3/projects/bb0c89cf3221414d9c616b2ba5d72477/users/0e77f97a88234d50a92ac14dd061dc92/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:56.444369 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0a5d0f32-1bfb-40dc-abfb-c8ed0aab437f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:56.454840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:56.460023 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d0f32-1bfb-40dc-abfb-c8ed0aab437f admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:56.462559 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41183326-5b1e-440e-a6e6-144e0198937c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb0c89cf3221414d9c616b2ba5d72477, user_id=0e77f97a88234d50a92ac14dd061dc92, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:56.466411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:56.467176 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:56.467336 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41183326-5b1e-440e-a6e6-144e0198937c admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:56.467661 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:56.468088 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4 admin admin] PATH_INFO: `/v3/projects/6162b281dea048c8b7c4c956c17c0c22` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:56.469742 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6162b281dea048c8b7c4c956c17c0c22)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:56.473341 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:56.479888 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:56.487154 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1407/2801] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:56 2022] PUT /identity/v3/projects/bb0c89cf3221414d9c616b2ba5d72477/users/0e77f97a88234d50a92ac14dd061dc92/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:56.498194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef150191-4098-452f-ad4c-78d418a8a7be None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:56.498479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef150191-4098-452f-ad4c-78d418a8a7be None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:56.498754 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef150191-4098-452f-ad4c-78d418a8a7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:56.525710 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6162b281dea048c8b7c4c956c17c0c22', 'request_id': 'req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:56.528305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-ef150191-4098-452f-ad4c-78d418a8a7be None None] MFA Rules not processed for user `0e77f97a88234d50a92ac14dd061dc92`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:56.529740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ef150191-4098-452f-ad4c-78d418a8a7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:56.533082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6162b281dea048c8b7c4c956c17c0c22', 'request_id': 'req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:56.539747 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6162b281dea048c8b7c4c956c17c0c22', 'request_id': 'req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:56.551602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4 admin admin] The token cache is being invalidate because project 6162b281dea048c8b7c4c956c17c0c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:56.551900 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:56.552578 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:56.554310 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1395/2802] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:56 2022] DELETE /identity/v3/projects/6162b281dea048c8b7c4c956c17c0c22 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:56.557999 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:56.559671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:56.596836 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1408/2803] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:56.608633 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef150191-4098-452f-ad4c-78d418a8a7be None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:56.610014 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ef150191-4098-452f-ad4c-78d418a8a7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:56.627737 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1025e30-a23e-4467-ad1d-6f6c2b7f4fb4 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:56.632334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2bca4-61af-462e-9890-a091726bf3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:56.633000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2bca4-61af-462e-9890-a091726bf3bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:56.633244 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2bca4-61af-462e-9890-a091726bf3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:56.633537 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2bca4-61af-462e-9890-a091726bf3bc admin admin] PATH_INFO: `/v3/users/69d5326c393343ef96c23a52f8c40f5a` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:56.634711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2bca4-61af-462e-9890-a091726bf3bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=69d5326c393343ef96c23a52f8c40f5a)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:56.636631 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2bca4-61af-462e-9890-a091726bf3bc admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:56.668333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef150191-4098-452f-ad4c-78d418a8a7be None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:56.672818 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2e36b-8a6b-455b-b0b3-06d1e25b862a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:56.673445 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2e36b-8a6b-455b-b0b3-06d1e25b862a service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:56.673723 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2e36b-8a6b-455b-b0b3-06d1e25b862a service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:56.673995 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2e36b-8a6b-455b-b0b3-06d1e25b862a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:56.685683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da2e36b-8a6b-455b-b0b3-06d1e25b862a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:56.687550 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da2e36b-8a6b-455b-b0b3-06d1e25b862a service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:56.690163 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-15b2bca4-61af-462e-9890-a091726bf3bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69d5326c393343ef96c23a52f8c40f5a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:56.699800 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-15b2bca4-61af-462e-9890-a091726bf3bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69d5326c393343ef96c23a52f8c40f5a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:56.700794 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-15b2bca4-61af-462e-9890-a091726bf3bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69d5326c393343ef96c23a52f8c40f5a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:56.701599 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-15b2bca4-61af-462e-9890-a091726bf3bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69d5326c393343ef96c23a52f8c40f5a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:56.704819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1409/2804] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:15:56.711870 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-15b2bca4-61af-462e-9890-a091726bf3bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69d5326c393343ef96c23a52f8c40f5a'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:56.730885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1396/2805] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:15:56 2022] DELETE /identity/v3/users/69d5326c393343ef96c23a52f8c40f5a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:15:57.016313 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2e36b-8a6b-455b-b0b3-06d1e25b862a service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:57.034787 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2e36b-8a6b-455b-b0b3-06d1e25b862a service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:57.100328 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99020465-4ec3-4647-96a3-d655c29b23e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:57.100962 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99020465-4ec3-4647-96a3-d655c29b23e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:57.101236 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99020465-4ec3-4647-96a3-d655c29b23e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:57.101515 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99020465-4ec3-4647-96a3-d655c29b23e4 admin admin] PATH_INFO: `/v3/projects/58490dcd2c6e44b1bac8a90b8f646934` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:57.102605 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99020465-4ec3-4647-96a3-d655c29b23e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58490dcd2c6e44b1bac8a90b8f646934)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:57.104564 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99020465-4ec3-4647-96a3-d655c29b23e4 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:57.280740 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:57.292031 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-99020465-4ec3-4647-96a3-d655c29b23e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58490dcd2c6e44b1bac8a90b8f646934', 'request_id': 'req-99020465-4ec3-4647-96a3-d655c29b23e4'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:57.301149 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-99020465-4ec3-4647-96a3-d655c29b23e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58490dcd2c6e44b1bac8a90b8f646934', 'request_id': 'req-99020465-4ec3-4647-96a3-d655c29b23e4'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:57.309205 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-99020465-4ec3-4647-96a3-d655c29b23e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58490dcd2c6e44b1bac8a90b8f646934', 'request_id': 'req-99020465-4ec3-4647-96a3-d655c29b23e4'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:57.321912 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-99020465-4ec3-4647-96a3-d655c29b23e4 admin admin] The token cache is being invalidate because project 58490dcd2c6e44b1bac8a90b8f646934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:15:57.322202 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-99020465-4ec3-4647-96a3-d655c29b23e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:57.322933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-99020465-4ec3-4647-96a3-d655c29b23e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:15:57.324578 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1410/2806] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:15:57 2022] DELETE /identity/v3/projects/58490dcd2c6e44b1bac8a90b8f646934 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:57.329730 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9185f40-8e09-4cce-bc39-1686955ae0a3 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:57.330201 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9185f40-8e09-4cce-bc39-1686955ae0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:57.330300 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9185f40-8e09-4cce-bc39-1686955ae0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:57.344263 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:57.350948 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-c9185f40-8e09-4cce-bc39-1686955ae0a3 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:57.352775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c9185f40-8e09-4cce-bc39-1686955ae0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:57.412115 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1397/2807] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:57.420954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbac500-1c4b-4e47-8920-de0ab5923a40 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:57.421222 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbac500-1c4b-4e47-8920-de0ab5923a40 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:57.421501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbac500-1c4b-4e47-8920-de0ab5923a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:57.442524 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-fbbac500-1c4b-4e47-8920-de0ab5923a40 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:57.443736 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-fbbac500-1c4b-4e47-8920-de0ab5923a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:57.452469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1411/2808] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:15:57.457031 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9185f40-8e09-4cce-bc39-1686955ae0a3 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:57.467151 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9185f40-8e09-4cce-bc39-1686955ae0a3 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:57.468474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbac500-1c4b-4e47-8920-de0ab5923a40 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:57.470018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-fbbac500-1c4b-4e47-8920-de0ab5923a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:57.472022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a52ddb-94e3-4681-8fc9-76bf253d8b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:57.472634 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a52ddb-94e3-4681-8fc9-76bf253d8b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:57.472914 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a52ddb-94e3-4681-8fc9-76bf253d8b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:57.473180 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a52ddb-94e3-4681-8fc9-76bf253d8b0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:57.473827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a52ddb-94e3-4681-8fc9-76bf253d8b0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:15:57.474130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a52ddb-94e3-4681-8fc9-76bf253d8b0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:57.475752 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a52ddb-94e3-4681-8fc9-76bf253d8b0f admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:57.488278 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1398/2809] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 16:15:57.492553 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a52ddb-94e3-4681-8fc9-76bf253d8b0f admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:57.502358 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a52ddb-94e3-4681-8fc9-76bf253d8b0f admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:57.507682 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ac95a-b387-46ed-8ff1-1340831113f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:57.509118 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ac95a-b387-46ed-8ff1-1340831113f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:57.509432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ac95a-b387-46ed-8ff1-1340831113f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:57.509673 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ac95a-b387-46ed-8ff1-1340831113f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:57.510318 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ac95a-b387-46ed-8ff1-1340831113f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:57.512502 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ac95a-b387-46ed-8ff1-1340831113f2 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:57.533334 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbac500-1c4b-4e47-8920-de0ab5923a40 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:57.538059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45008c-ab29-4cf0-af6c-aed83426b9cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:57.538661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45008c-ab29-4cf0-af6c-aed83426b9cf service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:57.538933 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45008c-ab29-4cf0-af6c-aed83426b9cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:57.539189 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45008c-ab29-4cf0-af6c-aed83426b9cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:57.541171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-bb45008c-ab29-4cf0-af6c-aed83426b9cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:57.541619 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1399/2810] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:15:57 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:57.545848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ac95a-b387-46ed-8ff1-1340831113f2 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:57.605142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45008c-ab29-4cf0-af6c-aed83426b9cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:57.606596 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45008c-ab29-4cf0-af6c-aed83426b9cf service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:57.610737 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ac95a-b387-46ed-8ff1-1340831113f2 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:57.615555 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150bc8d-194f-4224-a69e-a04689dccbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:57.616177 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150bc8d-194f-4224-a69e-a04689dccbda admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:57.616457 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150bc8d-194f-4224-a69e-a04689dccbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:57.616753 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3150bc8d-194f-4224-a69e-a04689dccbda admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:57.617395 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150bc8d-194f-4224-a69e-a04689dccbda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:57.619310 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3150bc8d-194f-4224-a69e-a04689dccbda admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:57.621616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1412/2811] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:15:57.645227 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1400/2812] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:15:57 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:15:57.646444 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45008c-ab29-4cf0-af6c-aed83426b9cf service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:57.657522 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45008c-ab29-4cf0-af6c-aed83426b9cf service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:57.662176 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670301f-8ac3-45cb-8af9-1258bb7d2971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:57.662862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670301f-8ac3-45cb-8af9-1258bb7d2971 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:57.663184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670301f-8ac3-45cb-8af9-1258bb7d2971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:57.663572 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670301f-8ac3-45cb-8af9-1258bb7d2971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:57.664215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5670301f-8ac3-45cb-8af9-1258bb7d2971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:57.665797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5670301f-8ac3-45cb-8af9-1258bb7d2971 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:57.678599 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1413/2813] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:57.680068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150bc8d-194f-4224-a69e-a04689dccbda admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:57.692666 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3150bc8d-194f-4224-a69e-a04689dccbda admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:57.698923 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e9fc7-8524-4b08-831a-89720e57a4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:57.699630 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e9fc7-8524-4b08-831a-89720e57a4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:57.700268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e9fc7-8524-4b08-831a-89720e57a4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:57.700268 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e9fc7-8524-4b08-831a-89720e57a4e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:57.700882 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e9fc7-8524-4b08-831a-89720e57a4e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:57.702953 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e9fc7-8524-4b08-831a-89720e57a4e7 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:57.721338 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1401/2814] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:15:57.725194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670301f-8ac3-45cb-8af9-1258bb7d2971 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:57.736386 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670301f-8ac3-45cb-8af9-1258bb7d2971 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:57.742146 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16b566-76ee-4ad5-b3d1-ec9205d02b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:57.743090 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16b566-76ee-4ad5-b3d1-ec9205d02b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:57.743359 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16b566-76ee-4ad5-b3d1-ec9205d02b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:57.743650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16b566-76ee-4ad5-b3d1-ec9205d02b25 admin admin] PATH_INFO: `/v3/projects/4b68c9423a4f46d0987ccde2779cdbce/users/c98f1bcbe3864db7ac873a8976f98696/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:57.760160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16b566-76ee-4ad5-b3d1-ec9205d02b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b68c9423a4f46d0987ccde2779cdbce, user_id=c98f1bcbe3864db7ac873a8976f98696, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:57.764791 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16b566-76ee-4ad5-b3d1-ec9205d02b25 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:57.802938 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1414/2815] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:15:57 2022] PUT /identity/v3/projects/4b68c9423a4f46d0987ccde2779cdbce/users/c98f1bcbe3864db7ac873a8976f98696/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:15:57.803342 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac936c3f-5148-4fad-8aa5-32c2aa7342ea None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:57.803693 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac936c3f-5148-4fad-8aa5-32c2aa7342ea None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:57.804007 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac936c3f-5148-4fad-8aa5-32c2aa7342ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:57.839020 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-ac936c3f-5148-4fad-8aa5-32c2aa7342ea None None] MFA Rules not processed for user `c98f1bcbe3864db7ac873a8976f98696`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:15:57.840397 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ac936c3f-5148-4fad-8aa5-32c2aa7342ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:57.880871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:57.907473 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1402/2816] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:15:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:15:57.919333 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16b566-76ee-4ad5-b3d1-ec9205d02b25 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:15:57.921272 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0c16b566-76ee-4ad5-b3d1-ec9205d02b25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:15:57.963261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:15:57.986673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16b566-76ee-4ad5-b3d1-ec9205d02b25 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:15:57.990535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faa37e-4e06-436d-b695-db9b8ea8b69c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f24ee16c043244938901c4e850dea967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:15:57.991175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faa37e-4e06-436d-b695-db9b8ea8b69c service glance] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:15:57.991498 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faa37e-4e06-436d-b695-db9b8ea8b69c service glance] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:15:57.991769 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faa37e-4e06-436d-b695-db9b8ea8b69c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:15:58.000180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faa37e-4e06-436d-b695-db9b8ea8b69c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:15:58.001633 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faa37e-4e06-436d-b695-db9b8ea8b69c service glance] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:15:58.011846 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1415/2817] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:15:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:16:06.683649 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a619b6-4725-4a0e-bf83-f92667001196 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:06.684250 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a619b6-4725-4a0e-bf83-f92667001196 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:06.684354 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a619b6-4725-4a0e-bf83-f92667001196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:06.705083 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-b0a619b6-4725-4a0e-bf83-f92667001196 None None] MFA Rules not processed for user `77aeaed3d55945e49b0eb6e3c0519294`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:06.706415 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-b0a619b6-4725-4a0e-bf83-f92667001196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:06.775014 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1403/2818] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:07.354974 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faa37e-4e06-436d-b695-db9b8ea8b69c service glance] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:07.356683 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c9faa37e-4e06-436d-b695-db9b8ea8b69c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:07.433903 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faa37e-4e06-436d-b695-db9b8ea8b69c service glance] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:07.441290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a51bf7-d5ef-429f-a909-94796d1f3fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:07.441290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a51bf7-d5ef-429f-a909-94796d1f3fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:07.441290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a51bf7-d5ef-429f-a909-94796d1f3fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:07.441290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a51bf7-d5ef-429f-a909-94796d1f3fdb admin admin] PATH_INFO: `/v3/users/0e77f97a88234d50a92ac14dd061dc92` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:07.442171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a51bf7-d5ef-429f-a909-94796d1f3fdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e77f97a88234d50a92ac14dd061dc92)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:07.444186 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a51bf7-d5ef-429f-a909-94796d1f3fdb admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:07.497678 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-26a51bf7-d5ef-429f-a909-94796d1f3fdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e77f97a88234d50a92ac14dd061dc92'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:07.508604 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-26a51bf7-d5ef-429f-a909-94796d1f3fdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e77f97a88234d50a92ac14dd061dc92'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:07.509275 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-26a51bf7-d5ef-429f-a909-94796d1f3fdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e77f97a88234d50a92ac14dd061dc92'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:07.509982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-26a51bf7-d5ef-429f-a909-94796d1f3fdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e77f97a88234d50a92ac14dd061dc92'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:07.524455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-26a51bf7-d5ef-429f-a909-94796d1f3fdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e77f97a88234d50a92ac14dd061dc92'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:07.540059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1416/2819] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:16:07 2022] DELETE /identity/v3/users/0e77f97a88234d50a92ac14dd061dc92 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:16:07.544990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4515a7e-9a1c-4d10-96c6-1779ecb806a7 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:07.545478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4515a7e-9a1c-4d10-96c6-1779ecb806a7 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:07.545797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4515a7e-9a1c-4d10-96c6-1779ecb806a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:07.569781 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-d4515a7e-9a1c-4d10-96c6-1779ecb806a7 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:07.571007 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d4515a7e-9a1c-4d10-96c6-1779ecb806a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:07.640198 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1404/2820] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:07.650817 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a51bf7-d5ef-429f-a909-94796d1f3fdb admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:07.652321 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-26a51bf7-d5ef-429f-a909-94796d1f3fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:07.726408 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a51bf7-d5ef-429f-a909-94796d1f3fdb admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:07.730739 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570196cd-7a34-498f-9622-97b58e833035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:07.731413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570196cd-7a34-498f-9622-97b58e833035 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:07.731735 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570196cd-7a34-498f-9622-97b58e833035 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:07.731964 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570196cd-7a34-498f-9622-97b58e833035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:07.745494 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570196cd-7a34-498f-9622-97b58e833035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:07.747095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570196cd-7a34-498f-9622-97b58e833035 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:07.752416 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:07.758277 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1417/2821] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:16:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:16:08.081189 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4515a7e-9a1c-4d10-96c6-1779ecb806a7 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:08.082995 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d4515a7e-9a1c-4d10-96c6-1779ecb806a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:08.089417 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570196cd-7a34-498f-9622-97b58e833035 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:08.091011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-570196cd-7a34-498f-9622-97b58e833035 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:08.117341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570196cd-7a34-498f-9622-97b58e833035 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:08.133889 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:08.134541 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:08.134811 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:08.135079 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c admin admin] PATH_INFO: `/v3/projects/bb0c89cf3221414d9c616b2ba5d72477` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:08.136405 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb0c89cf3221414d9c616b2ba5d72477)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:08.138208 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:08.198840 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb0c89cf3221414d9c616b2ba5d72477', 'request_id': 'req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:08.204280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4515a7e-9a1c-4d10-96c6-1779ecb806a7 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:08.207594 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb0c89cf3221414d9c616b2ba5d72477', 'request_id': 'req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:08.209304 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:08.216357 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb0c89cf3221414d9c616b2ba5d72477', 'request_id': 'req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:08.227671 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c admin admin] The token cache is being invalidate because project bb0c89cf3221414d9c616b2ba5d72477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:16:08.227968 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:08.228607 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:08.230152 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1418/2822] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:16:08 2022] DELETE /identity/v3/projects/bb0c89cf3221414d9c616b2ba5d72477 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:16:08.230902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:08.232169 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:08.272451 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7e7a8-df5e-4c07-bf3e-1a68cdb84f04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd28bb31b55e45668212c9ab69ed3ac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:08.273181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7e7a8-df5e-4c07-bf3e-1a68cdb84f04 service placement] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:08.273828 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7e7a8-df5e-4c07-bf3e-1a68cdb84f04 service placement] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:08.274105 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7e7a8-df5e-4c07-bf3e-1a68cdb84f04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:08.276547 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-adf7e7a8-df5e-4c07-bf3e-1a68cdb84f04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:08.288759 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7e7a8-df5e-4c07-bf3e-1a68cdb84f04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:08.290403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7e7a8-df5e-4c07-bf3e-1a68cdb84f04 service placement] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:08.295749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a953b95-1407-4dc4-87e2-3bc39e54fc6c admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:08.300320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2a840-62c4-472d-bf44-0c31783335f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:08.300885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1405/2823] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:16:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:16:08.301043 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2a840-62c4-472d-bf44-0c31783335f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:08.301231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2a840-62c4-472d-bf44-0c31783335f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:08.301555 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2a840-62c4-472d-bf44-0c31783335f8 admin admin] PATH_INFO: `/v3/users/bd619e168712480dac93519c4c469e71` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:08.302324 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3812e6-6fe9-4dbb-b31c-acdce12ad0e9 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:08.302602 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3812e6-6fe9-4dbb-b31c-acdce12ad0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:08.302730 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2a840-62c4-472d-bf44-0c31783335f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd619e168712480dac93519c4c469e71)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:08.302901 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3812e6-6fe9-4dbb-b31c-acdce12ad0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:08.304562 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2a840-62c4-472d-bf44-0c31783335f8 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:08.325896 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-bb3812e6-6fe9-4dbb-b31c-acdce12ad0e9 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:08.327541 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-bb3812e6-6fe9-4dbb-b31c-acdce12ad0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:08.337432 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1406/2824] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:08.353825 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601df45-6f23-4405-a935-712aa664c554 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:08.354235 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601df45-6f23-4405-a935-712aa664c554 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:08.354618 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601df45-6f23-4405-a935-712aa664c554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:08.361874 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-05e2a840-62c4-472d-bf44-0c31783335f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd619e168712480dac93519c4c469e71'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:08.374480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-05e2a840-62c4-472d-bf44-0c31783335f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd619e168712480dac93519c4c469e71'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:08.375339 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-05e2a840-62c4-472d-bf44-0c31783335f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd619e168712480dac93519c4c469e71'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:08.376087 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-05e2a840-62c4-472d-bf44-0c31783335f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd619e168712480dac93519c4c469e71'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:08.378316 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-c601df45-6f23-4405-a935-712aa664c554 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:08.379650 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-c601df45-6f23-4405-a935-712aa664c554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:08.391181 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1407/2825] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:08.392536 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-05e2a840-62c4-472d-bf44-0c31783335f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd619e168712480dac93519c4c469e71'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:08.395826 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601df45-6f23-4405-a935-712aa664c554 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:08.405677 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601df45-6f23-4405-a935-712aa664c554 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:08.410232 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7810209-4923-4333-8c07-de586ed8a407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:08.410886 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7810209-4923-4333-8c07-de586ed8a407 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:08.413922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7810209-4923-4333-8c07-de586ed8a407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:08.413922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7810209-4923-4333-8c07-de586ed8a407 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:08.413922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7810209-4923-4333-8c07-de586ed8a407 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:16:08.413922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7810209-4923-4333-8c07-de586ed8a407 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:08.414368 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1419/2826] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:16:08 2022] DELETE /identity/v3/users/bd619e168712480dac93519c4c469e71 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:16:08.414601 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7810209-4923-4333-8c07-de586ed8a407 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:08.416569 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a14d4-4167-4ac1-81ee-0d33610de87b None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:08.417736 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a14d4-4167-4ac1-81ee-0d33610de87b None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:08.418019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647a14d4-4167-4ac1-81ee-0d33610de87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:08.427132 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:08.434204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1408/2827] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:16:08.439089 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7810209-4923-4333-8c07-de586ed8a407 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:08.445515 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-647a14d4-4167-4ac1-81ee-0d33610de87b None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:08.447340 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-647a14d4-4167-4ac1-81ee-0d33610de87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:08.450563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7810209-4923-4333-8c07-de586ed8a407 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:08.515491 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1420/2828] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:08.520148 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6b396-8951-4a2e-9e3b-5cd0dadaed54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:08.520793 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6b396-8951-4a2e-9e3b-5cd0dadaed54 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:08.521099 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6b396-8951-4a2e-9e3b-5cd0dadaed54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:08.521370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6b396-8951-4a2e-9e3b-5cd0dadaed54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:08.522021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6b396-8951-4a2e-9e3b-5cd0dadaed54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:08.524096 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f6b396-8951-4a2e-9e3b-5cd0dadaed54 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:08.524781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a14d4-4167-4ac1-81ee-0d33610de87b None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:08.526156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-647a14d4-4167-4ac1-81ee-0d33610de87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:08.559226 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1409/2829] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:16:08 2022] POST /identity/v3/projects => generated 367 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:16:08.560572 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6b396-8951-4a2e-9e3b-5cd0dadaed54 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:08.592597 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647a14d4-4167-4ac1-81ee-0d33610de87b None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:08.629567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:08.634812 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6b396-8951-4a2e-9e3b-5cd0dadaed54 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:08.640627 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e11979-4295-4d18-ae28-6070e63b9e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:08.641366 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e11979-4295-4d18-ae28-6070e63b9e1e admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:08.641720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e11979-4295-4d18-ae28-6070e63b9e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:08.641991 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e11979-4295-4d18-ae28-6070e63b9e1e admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:08.642648 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e11979-4295-4d18-ae28-6070e63b9e1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:08.644591 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e11979-4295-4d18-ae28-6070e63b9e1e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:08.646399 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da2a27-5e09-482b-b4c1-403c18ee7310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:08.647003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da2a27-5e09-482b-b4c1-403c18ee7310 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:08.647271 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da2a27-5e09-482b-b4c1-403c18ee7310 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:08.647563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98da2a27-5e09-482b-b4c1-403c18ee7310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:08.656723 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da2a27-5e09-482b-b4c1-403c18ee7310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:08.658461 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98da2a27-5e09-482b-b4c1-403c18ee7310 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:08.667519 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1410/2830] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:16:08 2022] POST /identity/v3/users => generated 284 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:16:08.670902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1421/2831] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:16:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:16:08.671809 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e11979-4295-4d18-ae28-6070e63b9e1e admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:08.681720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e11979-4295-4d18-ae28-6070e63b9e1e admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:08.686828 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a214b-eab1-445e-8b21-e01a2c56a5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:08.687188 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a214b-eab1-445e-8b21-e01a2c56a5c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:08.687503 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a214b-eab1-445e-8b21-e01a2c56a5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:08.687782 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a214b-eab1-445e-8b21-e01a2c56a5c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:08.688524 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a214b-eab1-445e-8b21-e01a2c56a5c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:08.690367 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a214b-eab1-445e-8b21-e01a2c56a5c2 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:08.702606 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1411/2832] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:16:08.706999 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da2a27-5e09-482b-b4c1-403c18ee7310 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:08.718513 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98da2a27-5e09-482b-b4c1-403c18ee7310 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:08.724265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f655f3c-5f8d-4f8f-af16-88e51ac16352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:08.725023 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f655f3c-5f8d-4f8f-af16-88e51ac16352 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:08.725339 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f655f3c-5f8d-4f8f-af16-88e51ac16352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:08.725845 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f655f3c-5f8d-4f8f-af16-88e51ac16352 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:08.726574 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f655f3c-5f8d-4f8f-af16-88e51ac16352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:08.728490 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f655f3c-5f8d-4f8f-af16-88e51ac16352 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:08.739889 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1422/2833] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:16:08.744149 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a214b-eab1-445e-8b21-e01a2c56a5c2 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:08.757265 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a214b-eab1-445e-8b21-e01a2c56a5c2 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:08.761996 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50afa309-956a-44c4-8f3d-1edf57dcebc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:08.762777 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50afa309-956a-44c4-8f3d-1edf57dcebc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:08.763127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50afa309-956a-44c4-8f3d-1edf57dcebc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:08.763387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50afa309-956a-44c4-8f3d-1edf57dcebc1 admin admin] PATH_INFO: `/v3/projects/2f722f1a8cc74bee978fe23ce9b6bf1c/users/b0d93e4e3e404b1799d3c64e86cb7049/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:08.780676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50afa309-956a-44c4-8f3d-1edf57dcebc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f722f1a8cc74bee978fe23ce9b6bf1c, user_id=b0d93e4e3e404b1799d3c64e86cb7049, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:08.786977 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50afa309-956a-44c4-8f3d-1edf57dcebc1 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:08.801765 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1412/2834] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:16:08 2022] PUT /identity/v3/projects/2f722f1a8cc74bee978fe23ce9b6bf1c/users/b0d93e4e3e404b1799d3c64e86cb7049/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:16:08.809316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f9490-4ccd-43e1-a055-69088d4075e4 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:08.809978 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f9490-4ccd-43e1-a055-69088d4075e4 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:08.810406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f9490-4ccd-43e1-a055-69088d4075e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:08.842474 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-f56f9490-4ccd-43e1-a055-69088d4075e4 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:08.843769 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f56f9490-4ccd-43e1-a055-69088d4075e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:08.899113 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:08.918814 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1423/2835] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:08.929001 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50afa309-956a-44c4-8f3d-1edf57dcebc1 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:08.975642 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f9490-4ccd-43e1-a055-69088d4075e4 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:08.985216 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f56f9490-4ccd-43e1-a055-69088d4075e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:09.007016 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f9490-4ccd-43e1-a055-69088d4075e4 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:09.014910 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56373f25-8fdf-4178-bdad-1750b69d85a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:09.015331 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50afa309-956a-44c4-8f3d-1edf57dcebc1 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:09.015692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56373f25-8fdf-4178-bdad-1750b69d85a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:09.015974 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56373f25-8fdf-4178-bdad-1750b69d85a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:09.016248 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56373f25-8fdf-4178-bdad-1750b69d85a9 admin admin] PATH_INFO: `/v3/projects/8a6ef62e146a45ecb2c1d0b179713ed5` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:09.018400 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56373f25-8fdf-4178-bdad-1750b69d85a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a6ef62e146a45ecb2c1d0b179713ed5)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:09.020007 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f38ff-a303-4bef-bb45-882d4b0e6f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:09.020547 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56373f25-8fdf-4178-bdad-1750b69d85a9 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:09.021592 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f38ff-a303-4bef-bb45-882d4b0e6f90 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:09.021893 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f38ff-a303-4bef-bb45-882d4b0e6f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:09.022294 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f38ff-a303-4bef-bb45-882d4b0e6f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:09.034550 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f38ff-a303-4bef-bb45-882d4b0e6f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:09.036148 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f38ff-a303-4bef-bb45-882d4b0e6f90 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:09.049273 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1413/2836] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:16:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:16:09.101726 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-56373f25-8fdf-4178-bdad-1750b69d85a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a6ef62e146a45ecb2c1d0b179713ed5', 'request_id': 'req-56373f25-8fdf-4178-bdad-1750b69d85a9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:09.114320 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-56373f25-8fdf-4178-bdad-1750b69d85a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a6ef62e146a45ecb2c1d0b179713ed5', 'request_id': 'req-56373f25-8fdf-4178-bdad-1750b69d85a9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:09.125441 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-56373f25-8fdf-4178-bdad-1750b69d85a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a6ef62e146a45ecb2c1d0b179713ed5', 'request_id': 'req-56373f25-8fdf-4178-bdad-1750b69d85a9'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:09.161141 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-56373f25-8fdf-4178-bdad-1750b69d85a9 admin admin] The token cache is being invalidate because project 8a6ef62e146a45ecb2c1d0b179713ed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81526) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:16:09.161419 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-56373f25-8fdf-4178-bdad-1750b69d85a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:09.162290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-56373f25-8fdf-4178-bdad-1750b69d85a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:09.171796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1424/2837] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:16:08 2022] DELETE /identity/v3/projects/8a6ef62e146a45ecb2c1d0b179713ed5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:16:09.174466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6345d-4747-4ca2-84e7-4850f04b828f None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:09.175638 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6345d-4747-4ca2-84e7-4850f04b828f None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:09.176091 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6345d-4747-4ca2-84e7-4850f04b828f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:09.240162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-16a6345d-4747-4ca2-84e7-4850f04b828f None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:09.242411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-16a6345d-4747-4ca2-84e7-4850f04b828f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:09.348478 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1414/2838] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:09.357759 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f810fc9-99d6-452f-a3e6-37e6fa8eb6a7 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:09.358387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f810fc9-99d6-452f-a3e6-37e6fa8eb6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:09.359295 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f810fc9-99d6-452f-a3e6-37e6fa8eb6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:09.400336 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-2f810fc9-99d6-452f-a3e6-37e6fa8eb6a7 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:09.401775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-2f810fc9-99d6-452f-a3e6-37e6fa8eb6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:09.413649 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1425/2839] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:09.418325 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6345d-4747-4ca2-84e7-4850f04b828f None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:09.428829 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6345d-4747-4ca2-84e7-4850f04b828f None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:09.438912 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa6d29-326f-44d0-a196-edb322bbe376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:09.439687 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6d29-326f-44d0-a196-edb322bbe376 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:09.439963 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6d29-326f-44d0-a196-edb322bbe376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:09.440228 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa6d29-326f-44d0-a196-edb322bbe376 admin admin] PATH_INFO: `/v3/domains` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:09.440927 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa6d29-326f-44d0-a196-edb322bbe376 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81527) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:16:09.441226 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa6d29-326f-44d0-a196-edb322bbe376 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:09.443182 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa6d29-326f-44d0-a196-edb322bbe376 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:09.457635 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1415/2840] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:16:09.464227 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f810fc9-99d6-452f-a3e6-37e6fa8eb6a7 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:09.481168 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f810fc9-99d6-452f-a3e6-37e6fa8eb6a7 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:09.491932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc71ff-a71b-48e8-844b-225880163288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:09.493298 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc71ff-a71b-48e8-844b-225880163288 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:09.493620 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc71ff-a71b-48e8-844b-225880163288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:09.493954 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc71ff-a71b-48e8-844b-225880163288 admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:09.494703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc71ff-a71b-48e8-844b-225880163288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:09.496994 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc71ff-a71b-48e8-844b-225880163288 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:09.540243 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1426/2841] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:16:09 2022] POST /identity/v3/projects => generated 389 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:16:09.545234 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa6d29-326f-44d0-a196-edb322bbe376 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:09.560383 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa6d29-326f-44d0-a196-edb322bbe376 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:09.650681 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:09.680130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbbe15-d25f-4112-adb4-b7ed72b22633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:09.680983 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbbe15-d25f-4112-adb4-b7ed72b22633 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:09.681394 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbbe15-d25f-4112-adb4-b7ed72b22633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:09.681760 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bbbe15-d25f-4112-adb4-b7ed72b22633 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:09.682514 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbbe15-d25f-4112-adb4-b7ed72b22633 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:09.685065 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bbbe15-d25f-4112-adb4-b7ed72b22633 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:09.716908 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1416/2842] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:16:09 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:16:09.721391 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc71ff-a71b-48e8-844b-225880163288 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:09.735351 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc71ff-a71b-48e8-844b-225880163288 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:09.740616 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cfa91-d28e-417a-a6ac-3a3affe6cf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:09.741123 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cfa91-d28e-417a-a6ac-3a3affe6cf60 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:09.741397 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cfa91-d28e-417a-a6ac-3a3affe6cf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:09.741661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0cfa91-d28e-417a-a6ac-3a3affe6cf60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:09.742308 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cfa91-d28e-417a-a6ac-3a3affe6cf60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:09.743953 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0cfa91-d28e-417a-a6ac-3a3affe6cf60 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:09.756293 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1427/2843] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:16:09.761002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbbe15-d25f-4112-adb4-b7ed72b22633 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:09.771764 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bbbe15-d25f-4112-adb4-b7ed72b22633 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:09.776934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a99e6f8-136a-41ac-888c-405bd1994ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:09.777626 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99e6f8-136a-41ac-888c-405bd1994ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:09.777968 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99e6f8-136a-41ac-888c-405bd1994ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:09.778298 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a99e6f8-136a-41ac-888c-405bd1994ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:09.779040 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99e6f8-136a-41ac-888c-405bd1994ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:09.780890 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a99e6f8-136a-41ac-888c-405bd1994ac0 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:09.797804 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1417/2844] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:16:09.802289 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cfa91-d28e-417a-a6ac-3a3affe6cf60 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:09.813864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0cfa91-d28e-417a-a6ac-3a3affe6cf60 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:09.820660 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f10f57e-20f0-470f-8388-624a67e5015d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:09.821345 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10f57e-20f0-470f-8388-624a67e5015d admin admin] REQUEST_METHOD: `PUT` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:09.821620 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10f57e-20f0-470f-8388-624a67e5015d admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:09.821891 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f10f57e-20f0-470f-8388-624a67e5015d admin admin] PATH_INFO: `/v3/projects/d92d6dedd8af4159b87602b65363a198/users/32638e9f9c7e49acac58a002896c1114/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:09.855488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10f57e-20f0-470f-8388-624a67e5015d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d92d6dedd8af4159b87602b65363a198, user_id=32638e9f9c7e49acac58a002896c1114, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:09.859469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f10f57e-20f0-470f-8388-624a67e5015d admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:09.876230 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1428/2845] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:16:09 2022] PUT /identity/v3/projects/d92d6dedd8af4159b87602b65363a198/users/32638e9f9c7e49acac58a002896c1114/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:16:09.915692 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438121b-8cae-433b-88a6-1d3d539ab685 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:09.916042 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438121b-8cae-433b-88a6-1d3d539ab685 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:09.916240 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438121b-8cae-433b-88a6-1d3d539ab685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:09.953636 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8021390-e882-42dd-a7a9-a8f43ef2d792 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:09.953984 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8021390-e882-42dd-a7a9-a8f43ef2d792 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:09.954347 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8021390-e882-42dd-a7a9-a8f43ef2d792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:09.959997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-d438121b-8cae-433b-88a6-1d3d539ab685 None None] MFA Rules not processed for user `b0d93e4e3e404b1799d3c64e86cb7049`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:09.961208 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d438121b-8cae-433b-88a6-1d3d539ab685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:09.989019 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-c8021390-e882-42dd-a7a9-a8f43ef2d792 None None] MFA Rules not processed for user `32638e9f9c7e49acac58a002896c1114`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:09.991374 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c8021390-e882-42dd-a7a9-a8f43ef2d792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:10.053802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1418/2846] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:10.067276 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438121b-8cae-433b-88a6-1d3d539ab685 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:10.069629 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-d438121b-8cae-433b-88a6-1d3d539ab685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:10.088767 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:10.093419 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1429/2847] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:16:10.107657 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8021390-e882-42dd-a7a9-a8f43ef2d792 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:10.109940 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c8021390-e882-42dd-a7a9-a8f43ef2d792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:10.162468 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438121b-8cae-433b-88a6-1d3d539ab685 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:10.165280 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b684f8a8-76bb-437c-9846-d81c0bf6cf36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:10.166029 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684f8a8-76bb-437c-9846-d81c0bf6cf36 service cinder] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:10.166372 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684f8a8-76bb-437c-9846-d81c0bf6cf36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:10.167087 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b684f8a8-76bb-437c-9846-d81c0bf6cf36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:10.199244 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684f8a8-76bb-437c-9846-d81c0bf6cf36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:10.201370 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b684f8a8-76bb-437c-9846-d81c0bf6cf36 service cinder] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:10.213887 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1419/2848] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:16:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:16:10.226006 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8021390-e882-42dd-a7a9-a8f43ef2d792 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:10.230969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4658c4-502a-4de2-8273-e07f4f2c1e81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89df988d73ff48f3b7a92d335df9b633', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:10.231790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4658c4-502a-4de2-8273-e07f4f2c1e81 service cinder] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:10.232064 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4658c4-502a-4de2-8273-e07f4f2c1e81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:10.232373 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4658c4-502a-4de2-8273-e07f4f2c1e81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:10.242441 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4658c4-502a-4de2-8273-e07f4f2c1e81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:10.244095 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4658c4-502a-4de2-8273-e07f4f2c1e81 service cinder] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:10.258751 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1430/2849] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:16:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:16:11.687218 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb31684-754a-4acc-b8f6-f27cfa35ef32 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:11.687718 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb31684-754a-4acc-b8f6-f27cfa35ef32 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:11.687840 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb31684-754a-4acc-b8f6-f27cfa35ef32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:11.716857 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-5bb31684-754a-4acc-b8f6-f27cfa35ef32 None None] MFA Rules not processed for user `b0d93e4e3e404b1799d3c64e86cb7049`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:11.718763 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-5bb31684-754a-4acc-b8f6-f27cfa35ef32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:11.728074 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1420/2850] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:20.251426 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=26 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:21.701928 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:33.970346 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4658c4-502a-4de2-8273-e07f4f2c1e81 service cinder] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:34.043011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4658c4-502a-4de2-8273-e07f4f2c1e81 service cinder] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:34.047796 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3e650-7554-46d4-9880-cff603f57487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:34.048433 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3e650-7554-46d4-9880-cff603f57487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:34.048711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3e650-7554-46d4-9880-cff603f57487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:34.048980 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3e650-7554-46d4-9880-cff603f57487 admin admin] PATH_INFO: `/v3/users/32638e9f9c7e49acac58a002896c1114` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:34.050175 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3e650-7554-46d4-9880-cff603f57487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32638e9f9c7e49acac58a002896c1114)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:34.052237 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3e650-7554-46d4-9880-cff603f57487 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:34.106915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-4bf3e650-7554-46d4-9880-cff603f57487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32638e9f9c7e49acac58a002896c1114'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:34.115528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-4bf3e650-7554-46d4-9880-cff603f57487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32638e9f9c7e49acac58a002896c1114'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:34.116156 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-4bf3e650-7554-46d4-9880-cff603f57487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32638e9f9c7e49acac58a002896c1114'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:34.116824 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-4bf3e650-7554-46d4-9880-cff603f57487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32638e9f9c7e49acac58a002896c1114'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:34.130191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-4bf3e650-7554-46d4-9880-cff603f57487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32638e9f9c7e49acac58a002896c1114'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:34.143778 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1431/2851] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:16:33 2022] DELETE /identity/v3/users/32638e9f9c7e49acac58a002896c1114 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:16:34.148288 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd41a61-832e-44b5-ad2b-9a086db6cc5c None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:34.148764 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd41a61-832e-44b5-ad2b-9a086db6cc5c None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:34.148887 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd41a61-832e-44b5-ad2b-9a086db6cc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:34.167499 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-8dd41a61-832e-44b5-ad2b-9a086db6cc5c None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:34.168707 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-8dd41a61-832e-44b5-ad2b-9a086db6cc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:34.229379 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1421/2852] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:34.239528 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3e650-7554-46d4-9880-cff603f57487 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:34.241029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-4bf3e650-7554-46d4-9880-cff603f57487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:34.320487 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3e650-7554-46d4-9880-cff603f57487 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:34.324630 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10736c-054e-4816-8d39-316efaf316bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:34.325231 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10736c-054e-4816-8d39-316efaf316bc service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:34.325511 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10736c-054e-4816-8d39-316efaf316bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:34.325776 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10736c-054e-4816-8d39-316efaf316bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:34.336201 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10736c-054e-4816-8d39-316efaf316bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:34.337647 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10736c-054e-4816-8d39-316efaf316bc service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:34.347709 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1432/2853] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:16:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:16:34.717408 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd41a61-832e-44b5-ad2b-9a086db6cc5c None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:34.719058 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-8dd41a61-832e-44b5-ad2b-9a086db6cc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:34.732832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd41a61-832e-44b5-ad2b-9a086db6cc5c None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:34.744129 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cbbe78-44f3-454a-8ba0-dfe67bef0f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:34.744797 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbbe78-44f3-454a-8ba0-dfe67bef0f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:34.745125 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbbe78-44f3-454a-8ba0-dfe67bef0f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:34.745402 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cbbe78-44f3-454a-8ba0-dfe67bef0f3e admin admin] PATH_INFO: `/v3/projects/d92d6dedd8af4159b87602b65363a198` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:34.747068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbbe78-44f3-454a-8ba0-dfe67bef0f3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d92d6dedd8af4159b87602b65363a198)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:34.748565 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cbbe78-44f3-454a-8ba0-dfe67bef0f3e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:34.796906 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-52cbbe78-44f3-454a-8ba0-dfe67bef0f3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd92d6dedd8af4159b87602b65363a198', 'request_id': 'req-52cbbe78-44f3-454a-8ba0-dfe67bef0f3e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:34.805259 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-52cbbe78-44f3-454a-8ba0-dfe67bef0f3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd92d6dedd8af4159b87602b65363a198', 'request_id': 'req-52cbbe78-44f3-454a-8ba0-dfe67bef0f3e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:34.811518 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-52cbbe78-44f3-454a-8ba0-dfe67bef0f3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd92d6dedd8af4159b87602b65363a198', 'request_id': 'req-52cbbe78-44f3-454a-8ba0-dfe67bef0f3e'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:34.824465 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-52cbbe78-44f3-454a-8ba0-dfe67bef0f3e admin admin] The token cache is being invalidate because project d92d6dedd8af4159b87602b65363a198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:16:34.824752 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-52cbbe78-44f3-454a-8ba0-dfe67bef0f3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:34.825466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-52cbbe78-44f3-454a-8ba0-dfe67bef0f3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:34.827084 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1422/2854] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:16:34 2022] DELETE /identity/v3/projects/d92d6dedd8af4159b87602b65363a198 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:16:34.832305 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cafbe-ccfd-4e5a-abf3-c4904841133f None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:34.832590 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cafbe-ccfd-4e5a-abf3-c4904841133f None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:34.832864 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cafbe-ccfd-4e5a-abf3-c4904841133f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:34.841224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:34.854951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-0b1cafbe-ccfd-4e5a-abf3-c4904841133f None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:34.856961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-0b1cafbe-ccfd-4e5a-abf3-c4904841133f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:34.953650 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1433/2855] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 16:16:34.961445 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943fc2f-9aaf-4e2f-874a-dac96490c133 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:34.961729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943fc2f-9aaf-4e2f-874a-dac96490c133 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:34.962000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943fc2f-9aaf-4e2f-874a-dac96490c133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:34.982328 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-3943fc2f-9aaf-4e2f-874a-dac96490c133 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:34.983550 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-3943fc2f-9aaf-4e2f-874a-dac96490c133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:34.992337 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1423/2856] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:34.996363 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1cafbe-ccfd-4e5a-abf3-c4904841133f None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:35.006687 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1cafbe-ccfd-4e5a-abf3-c4904841133f None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:35.011323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65240f90-18fd-43c9-bf71-57693d8729ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:35.011965 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65240f90-18fd-43c9-bf71-57693d8729ae admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:35.012238 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65240f90-18fd-43c9-bf71-57693d8729ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:35.012504 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65240f90-18fd-43c9-bf71-57693d8729ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:35.013184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65240f90-18fd-43c9-bf71-57693d8729ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:16:35.013449 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65240f90-18fd-43c9-bf71-57693d8729ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:35.015039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65240f90-18fd-43c9-bf71-57693d8729ae admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:35.027391 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1434/2857] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:16:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:16:35.032008 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943fc2f-9aaf-4e2f-874a-dac96490c133 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:35.041168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943fc2f-9aaf-4e2f-874a-dac96490c133 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:35.045889 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52992623-daf4-428a-9901-03370f13223e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:35.046511 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52992623-daf4-428a-9901-03370f13223e admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:35.046822 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52992623-daf4-428a-9901-03370f13223e admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:35.047158 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52992623-daf4-428a-9901-03370f13223e admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:35.047751 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52992623-daf4-428a-9901-03370f13223e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:35.049643 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52992623-daf4-428a-9901-03370f13223e admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:35.079165 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1424/2858] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:16:35 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:16:35.083771 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65240f90-18fd-43c9-bf71-57693d8729ae admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:35.094159 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65240f90-18fd-43c9-bf71-57693d8729ae admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:35.154057 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21901572-2634-499f-ba0a-18c78881ec4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:35.155011 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21901572-2634-499f-ba0a-18c78881ec4c admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:35.155481 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21901572-2634-499f-ba0a-18c78881ec4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:35.155946 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21901572-2634-499f-ba0a-18c78881ec4c admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:35.156979 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21901572-2634-499f-ba0a-18c78881ec4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:35.160283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21901572-2634-499f-ba0a-18c78881ec4c admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:35.192418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1435/2859] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:16:35 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:16:35.197036 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52992623-daf4-428a-9901-03370f13223e admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:35.211605 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52992623-daf4-428a-9901-03370f13223e admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:35.218903 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ae71a-3cae-40e4-9e04-8a4aeaa6bb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:35.219563 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ae71a-3cae-40e4-9e04-8a4aeaa6bb66 admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:35.219841 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ae71a-3cae-40e4-9e04-8a4aeaa6bb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:35.220184 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74ae71a-3cae-40e4-9e04-8a4aeaa6bb66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:35.220859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ae71a-3cae-40e4-9e04-8a4aeaa6bb66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:35.222553 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74ae71a-3cae-40e4-9e04-8a4aeaa6bb66 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:35.234000 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1425/2860] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:16:35.238354 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21901572-2634-499f-ba0a-18c78881ec4c admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:35.249503 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21901572-2634-499f-ba0a-18c78881ec4c admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:35.254639 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5858e851-52d0-4a59-ad85-201ed55fe603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:35.255335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e851-52d0-4a59-ad85-201ed55fe603 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:35.255876 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e851-52d0-4a59-ad85-201ed55fe603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:35.256144 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5858e851-52d0-4a59-ad85-201ed55fe603 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:35.256795 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858e851-52d0-4a59-ad85-201ed55fe603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:35.258409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5858e851-52d0-4a59-ad85-201ed55fe603 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:35.269718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1436/2861] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:16:35.274168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ae71a-3cae-40e4-9e04-8a4aeaa6bb66 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:35.287121 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74ae71a-3cae-40e4-9e04-8a4aeaa6bb66 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:35.296122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829ce3e-8f53-40bf-8e31-f93635b579d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:35.296983 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829ce3e-8f53-40bf-8e31-f93635b579d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:35.297414 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829ce3e-8f53-40bf-8e31-f93635b579d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:35.297594 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829ce3e-8f53-40bf-8e31-f93635b579d0 admin admin] PATH_INFO: `/v3/projects/223d0d2bc7a54f5b9ccc4b4414604d87/users/d87960cdc11f47e2ac431484d3a56bd2/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:35.317072 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829ce3e-8f53-40bf-8e31-f93635b579d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=223d0d2bc7a54f5b9ccc4b4414604d87, user_id=d87960cdc11f47e2ac431484d3a56bd2, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:35.319785 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829ce3e-8f53-40bf-8e31-f93635b579d0 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:35.335651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1426/2862] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:16:35 2022] PUT /identity/v3/projects/223d0d2bc7a54f5b9ccc4b4414604d87/users/d87960cdc11f47e2ac431484d3a56bd2/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:16:35.341245 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2e4dc-22a9-457c-ba1c-5b27e59e1220 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:35.341553 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2e4dc-22a9-457c-ba1c-5b27e59e1220 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:35.341824 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c2e4dc-22a9-457c-ba1c-5b27e59e1220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:35.361662 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-62c2e4dc-22a9-457c-ba1c-5b27e59e1220 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:35.362867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-62c2e4dc-22a9-457c-ba1c-5b27e59e1220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:35.433038 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1437/2863] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:35.442959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829ce3e-8f53-40bf-8e31-f93635b579d0 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:35.444836 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-0829ce3e-8f53-40bf-8e31-f93635b579d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:35.468334 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:35.507574 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829ce3e-8f53-40bf-8e31-f93635b579d0 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:35.512379 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd056ee-ebc9-45ea-a66e-c4ec2713ccb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:35.512996 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd056ee-ebc9-45ea-a66e-c4ec2713ccb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:35.513274 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd056ee-ebc9-45ea-a66e-c4ec2713ccb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:35.513546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd056ee-ebc9-45ea-a66e-c4ec2713ccb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:35.524170 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd056ee-ebc9-45ea-a66e-c4ec2713ccb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:35.525609 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd056ee-ebc9-45ea-a66e-c4ec2713ccb1 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:35.536092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1427/2864] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:16:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:16:37.326832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2e4dc-22a9-457c-ba1c-5b27e59e1220 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:37.329048 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-62c2e4dc-22a9-457c-ba1c-5b27e59e1220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:37.345672 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c2e4dc-22a9-457c-ba1c-5b27e59e1220 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:37.374161 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019570b4-a268-43d6-89bf-fccede90865b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:37.374835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019570b4-a268-43d6-89bf-fccede90865b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:37.375106 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019570b4-a268-43d6-89bf-fccede90865b admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:37.375370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019570b4-a268-43d6-89bf-fccede90865b admin admin] PATH_INFO: `/v3/users/c98f1bcbe3864db7ac873a8976f98696` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:37.380454 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019570b4-a268-43d6-89bf-fccede90865b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c98f1bcbe3864db7ac873a8976f98696)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:37.386645 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019570b4-a268-43d6-89bf-fccede90865b admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:37.396143 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:37.503418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-019570b4-a268-43d6-89bf-fccede90865b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c98f1bcbe3864db7ac873a8976f98696'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:37.530318 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-019570b4-a268-43d6-89bf-fccede90865b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c98f1bcbe3864db7ac873a8976f98696'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:37.530991 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-019570b4-a268-43d6-89bf-fccede90865b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c98f1bcbe3864db7ac873a8976f98696'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:37.531661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-019570b4-a268-43d6-89bf-fccede90865b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c98f1bcbe3864db7ac873a8976f98696'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:37.554062 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-019570b4-a268-43d6-89bf-fccede90865b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c98f1bcbe3864db7ac873a8976f98696'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:37.584996 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1438/2865] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:16:37 2022] DELETE /identity/v3/users/c98f1bcbe3864db7ac873a8976f98696 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:16:37.587168 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f115b7e-ecad-4c5b-81a1-862cff9a15ac None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:37.587679 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f115b7e-ecad-4c5b-81a1-862cff9a15ac None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:37.587784 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f115b7e-ecad-4c5b-81a1-862cff9a15ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:37.648981 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-9f115b7e-ecad-4c5b-81a1-862cff9a15ac None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:37.652094 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9f115b7e-ecad-4c5b-81a1-862cff9a15ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:37.746561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1428/2866] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:37.766322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019570b4-a268-43d6-89bf-fccede90865b admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:37.768664 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-019570b4-a268-43d6-89bf-fccede90865b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:37.854179 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019570b4-a268-43d6-89bf-fccede90865b admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:37.858354 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd35ac-a990-4e09-943b-9ce107473164 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:37.858978 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd35ac-a990-4e09-943b-9ce107473164 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:37.859250 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd35ac-a990-4e09-943b-9ce107473164 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:37.859567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cd35ac-a990-4e09-943b-9ce107473164 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:37.878304 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd35ac-a990-4e09-943b-9ce107473164 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:37.879904 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cd35ac-a990-4e09-943b-9ce107473164 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:37.896496 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1439/2867] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:16:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:16:38.435386 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f115b7e-ecad-4c5b-81a1-862cff9a15ac None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:38.436994 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-9f115b7e-ecad-4c5b-81a1-862cff9a15ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:38.455561 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f115b7e-ecad-4c5b-81a1-862cff9a15ac None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:38.472443 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd241d-9a8d-4278-8ca7-5d9cf7ad79b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:38.473380 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd241d-9a8d-4278-8ca7-5d9cf7ad79b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:38.473729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd241d-9a8d-4278-8ca7-5d9cf7ad79b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:38.474112 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd241d-9a8d-4278-8ca7-5d9cf7ad79b5 admin admin] PATH_INFO: `/v3/projects/4b68c9423a4f46d0987ccde2779cdbce` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:38.476003 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd241d-9a8d-4278-8ca7-5d9cf7ad79b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b68c9423a4f46d0987ccde2779cdbce)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:38.478165 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd241d-9a8d-4278-8ca7-5d9cf7ad79b5 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:38.586712 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-12dd241d-9a8d-4278-8ca7-5d9cf7ad79b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b68c9423a4f46d0987ccde2779cdbce', 'request_id': 'req-12dd241d-9a8d-4278-8ca7-5d9cf7ad79b5'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:38.614086 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-12dd241d-9a8d-4278-8ca7-5d9cf7ad79b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b68c9423a4f46d0987ccde2779cdbce', 'request_id': 'req-12dd241d-9a8d-4278-8ca7-5d9cf7ad79b5'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:38.634520 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-12dd241d-9a8d-4278-8ca7-5d9cf7ad79b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b68c9423a4f46d0987ccde2779cdbce', 'request_id': 'req-12dd241d-9a8d-4278-8ca7-5d9cf7ad79b5'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:38.663165 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-12dd241d-9a8d-4278-8ca7-5d9cf7ad79b5 admin admin] The token cache is being invalidate because project 4b68c9423a4f46d0987ccde2779cdbce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:16:38.663967 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-12dd241d-9a8d-4278-8ca7-5d9cf7ad79b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:38.667101 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-12dd241d-9a8d-4278-8ca7-5d9cf7ad79b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:38.674291 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1429/2868] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:16:38 2022] DELETE /identity/v3/projects/4b68c9423a4f46d0987ccde2779cdbce => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:16:41.512691 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd35ac-a990-4e09-943b-9ce107473164 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:41.542973 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cd35ac-a990-4e09-943b-9ce107473164 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:41.665481 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed39f5e-8426-43f6-87b7-82ade2d7b716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:41.666283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed39f5e-8426-43f6-87b7-82ade2d7b716 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:41.666567 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed39f5e-8426-43f6-87b7-82ade2d7b716 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:41.667244 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed39f5e-8426-43f6-87b7-82ade2d7b716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:41.669191 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6ed39f5e-8426-43f6-87b7-82ade2d7b716 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:41.726367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:41.764514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed39f5e-8426-43f6-87b7-82ade2d7b716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:41.766547 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed39f5e-8426-43f6-87b7-82ade2d7b716 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:41.779157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1440/2869] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:16:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:16:41.781219 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c1099-979d-4bc1-b55e-2ff38bc63bc7 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:41.781701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c1099-979d-4bc1-b55e-2ff38bc63bc7 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:41.783700 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c1099-979d-4bc1-b55e-2ff38bc63bc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:41.831215 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-e26c1099-979d-4bc1-b55e-2ff38bc63bc7 None None] MFA Rules not processed for user `d87960cdc11f47e2ac431484d3a56bd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:41.833803 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-e26c1099-979d-4bc1-b55e-2ff38bc63bc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:41.877436 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:41.920059 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1430/2870] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:41.924718 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed39f5e-8426-43f6-87b7-82ade2d7b716 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:41.926125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-6ed39f5e-8426-43f6-87b7-82ade2d7b716 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:41.939126 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed39f5e-8426-43f6-87b7-82ade2d7b716 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:41.944166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ca440-1b48-4eed-b158-7b807727e25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:41.944778 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ca440-1b48-4eed-b158-7b807727e25a admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:41.945051 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ca440-1b48-4eed-b158-7b807727e25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:41.945352 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ca440-1b48-4eed-b158-7b807727e25a admin admin] PATH_INFO: `/v3/projects` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:41.945998 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ca440-1b48-4eed-b158-7b807727e25a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:41.947982 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ca440-1b48-4eed-b158-7b807727e25a admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:41.976026 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1441/2871] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:16:41 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:16:41.981120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c1099-979d-4bc1-b55e-2ff38bc63bc7 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:41.992466 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c1099-979d-4bc1-b55e-2ff38bc63bc7 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:42.052068 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92329415-a6a0-492b-b13a-4ac558e2b187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:42.052745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92329415-a6a0-492b-b13a-4ac558e2b187 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:42.053100 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92329415-a6a0-492b-b13a-4ac558e2b187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:42.053441 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92329415-a6a0-492b-b13a-4ac558e2b187 admin admin] PATH_INFO: `/v3/users` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:42.054175 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92329415-a6a0-492b-b13a-4ac558e2b187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:42.056243 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92329415-a6a0-492b-b13a-4ac558e2b187 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:42.079072 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1431/2872] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:16:41 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 16:16:42.083037 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ca440-1b48-4eed-b158-7b807727e25a admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:42.093273 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ca440-1b48-4eed-b158-7b807727e25a admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:42.097835 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8be40-15cd-4685-9234-af57a4bad348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:42.098441 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8be40-15cd-4685-9234-af57a4bad348 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:42.098720 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8be40-15cd-4685-9234-af57a4bad348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:42.098993 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8be40-15cd-4685-9234-af57a4bad348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:42.099673 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8be40-15cd-4685-9234-af57a4bad348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:42.101290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8be40-15cd-4685-9234-af57a4bad348 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:42.115506 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1442/2873] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 16:16:42.117092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92329415-a6a0-492b-b13a-4ac558e2b187 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:42.127746 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92329415-a6a0-492b-b13a-4ac558e2b187 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:42.132463 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4384d-81c0-45e0-a42d-75423c35a0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:42.133123 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4384d-81c0-45e0-a42d-75423c35a0a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:42.133398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4384d-81c0-45e0-a42d-75423c35a0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:42.133668 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4384d-81c0-45e0-a42d-75423c35a0a8 admin admin] PATH_INFO: `/v3/projects/32e079fde1ca49f3a2bbcf5da1d35d6d/users/c1c6b313f3a1481a92ad373f25504e55/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:42.150063 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4384d-81c0-45e0-a42d-75423c35a0a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32e079fde1ca49f3a2bbcf5da1d35d6d, user_id=c1c6b313f3a1481a92ad373f25504e55, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:42.152874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4384d-81c0-45e0-a42d-75423c35a0a8 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:42.167816 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1432/2874] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:16:42 2022] PUT /identity/v3/projects/32e079fde1ca49f3a2bbcf5da1d35d6d/users/c1c6b313f3a1481a92ad373f25504e55/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 16:16:42.171883 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8be40-15cd-4685-9234-af57a4bad348 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:42.188204 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8be40-15cd-4685-9234-af57a4bad348 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:42.252988 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b0895-0659-41d3-9953-bbb6dce79ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:42.253707 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b0895-0659-41d3-9953-bbb6dce79ddc admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:42.254129 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b0895-0659-41d3-9953-bbb6dce79ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:42.254489 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b0895-0659-41d3-9953-bbb6dce79ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:42.255046 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b0895-0659-41d3-9953-bbb6dce79ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:42.256693 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b0895-0659-41d3-9953-bbb6dce79ddc admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:42.270834 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1443/2875] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:16:42.273956 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4384d-81c0-45e0-a42d-75423c35a0a8 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:42.288461 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4384d-81c0-45e0-a42d-75423c35a0a8 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:42.293671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836ef88-7d87-47b2-a083-907c8419f332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:42.294343 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836ef88-7d87-47b2-a083-907c8419f332 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:42.294609 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836ef88-7d87-47b2-a083-907c8419f332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:42.294873 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836ef88-7d87-47b2-a083-907c8419f332 admin admin] PATH_INFO: `/v3/projects/32e079fde1ca49f3a2bbcf5da1d35d6d/users/c1c6b313f3a1481a92ad373f25504e55/roles/bb69e9f6c5ed4649a2810514178a6216` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:42.297609 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836ef88-7d87-47b2-a083-907c8419f332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32e079fde1ca49f3a2bbcf5da1d35d6d, user_id=c1c6b313f3a1481a92ad373f25504e55, role_id=bb69e9f6c5ed4649a2810514178a6216)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:42.300670 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836ef88-7d87-47b2-a083-907c8419f332 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:42.313462 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1433/2876] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:16:42 2022] PUT /identity/v3/projects/32e079fde1ca49f3a2bbcf5da1d35d6d/users/c1c6b313f3a1481a92ad373f25504e55/roles/bb69e9f6c5ed4649a2810514178a6216 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:16:43.343777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b0895-0659-41d3-9953-bbb6dce79ddc admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:43.345779 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ce0b0895-0659-41d3-9953-bbb6dce79ddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:43.470367 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b0895-0659-41d3-9953-bbb6dce79ddc admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:43.537563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0300c8-7799-45d0-b941-a4c29b1eb747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:43.537858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0300c8-7799-45d0-b941-a4c29b1eb747 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:43.538973 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0300c8-7799-45d0-b941-a4c29b1eb747 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:43.538973 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0300c8-7799-45d0-b941-a4c29b1eb747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:43.540634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-7e0300c8-7799-45d0-b941-a4c29b1eb747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:43.548066 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:43.647161 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0300c8-7799-45d0-b941-a4c29b1eb747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:43.649736 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0300c8-7799-45d0-b941-a4c29b1eb747 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:43.663487 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1444/2877] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:16:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:16:45.294660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7852a4-3a4a-46f0-a41a-f7799b960d79 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:45.294660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7852a4-3a4a-46f0-a41a-f7799b960d79 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:45.294660 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7852a4-3a4a-46f0-a41a-f7799b960d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:45.351465 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-dd7852a4-3a4a-46f0-a41a-f7799b960d79 None None] MFA Rules not processed for user `b0d93e4e3e404b1799d3c64e86cb7049`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:45.353126 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-dd7852a4-3a4a-46f0-a41a-f7799b960d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:45.501827 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1434/2878] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:50.876341 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53655e91-a43c-468b-919a-52c4f489f536 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:50.876872 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53655e91-a43c-468b-919a-52c4f489f536 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:50.876966 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53655e91-a43c-468b-919a-52c4f489f536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:50.917581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-53655e91-a43c-468b-919a-52c4f489f536 None None] MFA Rules not processed for user `c1c6b313f3a1481a92ad373f25504e55`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:50.918827 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-53655e91-a43c-468b-919a-52c4f489f536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:51.018530 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1445/2879] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:16:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:51.024992 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7852a4-3a4a-46f0-a41a-f7799b960d79 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:51.026517 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-dd7852a4-3a4a-46f0-a41a-f7799b960d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:51.068062 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:51.097122 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7852a4-3a4a-46f0-a41a-f7799b960d79 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:51.100987 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f73e1d-e2b8-4717-9b7c-39374bdb1d6b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f24ee16c043244938901c4e850dea967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:51.101974 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f73e1d-e2b8-4717-9b7c-39374bdb1d6b service glance] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:51.102722 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f73e1d-e2b8-4717-9b7c-39374bdb1d6b service glance] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:51.103064 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f73e1d-e2b8-4717-9b7c-39374bdb1d6b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:51.203603 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f73e1d-e2b8-4717-9b7c-39374bdb1d6b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:51.205074 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f73e1d-e2b8-4717-9b7c-39374bdb1d6b service glance] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:51.214640 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1435/2880] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:16:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 16:16:51.364163 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194e1b1-8e08-473c-a3a3-74b6505d6acc None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:51.364442 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194e1b1-8e08-473c-a3a3-74b6505d6acc None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:51.364711 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c194e1b1-8e08-473c-a3a3-74b6505d6acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:51.396870 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-c194e1b1-8e08-473c-a3a3-74b6505d6acc None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:16:51.398224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c194e1b1-8e08-473c-a3a3-74b6505d6acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:51.466171 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1446/2881] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Sep 8 16:16:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:16:51.518697 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f73e1d-e2b8-4717-9b7c-39374bdb1d6b service glance] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:51.520770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-95f73e1d-e2b8-4717-9b7c-39374bdb1d6b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:51.600199 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f73e1d-e2b8-4717-9b7c-39374bdb1d6b service glance] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:51.606111 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df3a3-9faf-4571-ad4d-31879839d69e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:51.606732 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df3a3-9faf-4571-ad4d-31879839d69e service swift] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:51.607146 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df3a3-9faf-4571-ad4d-31879839d69e service swift] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:51.607278 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5df3a3-9faf-4571-ad4d-31879839d69e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:51.616375 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5df3a3-9faf-4571-ad4d-31879839d69e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:51.617990 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5df3a3-9faf-4571-ad4d-31879839d69e service swift] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:51.629048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1436/2882] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:16:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:16:53.528309 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194e1b1-8e08-473c-a3a3-74b6505d6acc None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:53.529788 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-c194e1b1-8e08-473c-a3a3-74b6505d6acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:53.585029 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:53.726022 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c194e1b1-8e08-473c-a3a3-74b6505d6acc None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:53.738299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7424b-3897-4cc7-a038-51ebcb7ab829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:53.738968 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7424b-3897-4cc7-a038-51ebcb7ab829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:53.739240 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7424b-3897-4cc7-a038-51ebcb7ab829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:53.739585 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd7424b-3897-4cc7-a038-51ebcb7ab829 admin admin] PATH_INFO: `/v3/users/f3be588325bf40beae354c1237d8e9ea` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:53.740866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7424b-3897-4cc7-a038-51ebcb7ab829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3be588325bf40beae354c1237d8e9ea)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:53.742825 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd7424b-3897-4cc7-a038-51ebcb7ab829 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:53.849733 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3fd7424b-3897-4cc7-a038-51ebcb7ab829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3be588325bf40beae354c1237d8e9ea'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:53.868300 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3fd7424b-3897-4cc7-a038-51ebcb7ab829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3be588325bf40beae354c1237d8e9ea'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:53.869246 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3fd7424b-3897-4cc7-a038-51ebcb7ab829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3be588325bf40beae354c1237d8e9ea'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:53.870749 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3fd7424b-3897-4cc7-a038-51ebcb7ab829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3be588325bf40beae354c1237d8e9ea'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:53.916971 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-3fd7424b-3897-4cc7-a038-51ebcb7ab829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3be588325bf40beae354c1237d8e9ea'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:53.946610 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1447/2883] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:16:53 2022] DELETE /identity/v3/users/f3be588325bf40beae354c1237d8e9ea => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:16:54.207971 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df3a3-9faf-4571-ad4d-31879839d69e service swift] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:54.232866 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5df3a3-9faf-4571-ad4d-31879839d69e service swift] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:54.355677 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:54.356546 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:54.356709 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:54.356832 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d admin admin] PATH_INFO: `/v3/projects/2245b7557087417bb8a32322c8673cef` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:54.361045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2245b7557087417bb8a32322c8673cef)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:54.367114 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:54.396873 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:54.459853 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2245b7557087417bb8a32322c8673cef', 'request_id': 'req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:54.475393 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2245b7557087417bb8a32322c8673cef', 'request_id': 'req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:54.492859 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2245b7557087417bb8a32322c8673cef', 'request_id': 'req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:54.522916 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d admin admin] The token cache is being invalidate because project 2245b7557087417bb8a32322c8673cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:16:54.526130 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:54.527298 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:54.529494 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1437/2884] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:16:54 2022] DELETE /identity/v3/projects/2245b7557087417bb8a32322c8673cef => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:16:54.533713 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7424b-3897-4cc7-a038-51ebcb7ab829 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:54.535666 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3fd7424b-3897-4cc7-a038-51ebcb7ab829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:54.553479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd7424b-3897-4cc7-a038-51ebcb7ab829 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:54.626267 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eae633-2fce-4f7c-aebd-cf45f6fe03cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:54.626856 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eae633-2fce-4f7c-aebd-cf45f6fe03cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:54.627136 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eae633-2fce-4f7c-aebd-cf45f6fe03cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:54.627406 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eae633-2fce-4f7c-aebd-cf45f6fe03cf admin admin] PATH_INFO: `/v3/users/f27ce8a59f7a463e9106dd8f6ad0191b` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:54.628619 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eae633-2fce-4f7c-aebd-cf45f6fe03cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f27ce8a59f7a463e9106dd8f6ad0191b)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:54.630720 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eae633-2fce-4f7c-aebd-cf45f6fe03cf admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:54.694909 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:16:54.704744 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d1eae633-2fce-4f7c-aebd-cf45f6fe03cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f27ce8a59f7a463e9106dd8f6ad0191b'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:54.725317 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d1eae633-2fce-4f7c-aebd-cf45f6fe03cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f27ce8a59f7a463e9106dd8f6ad0191b'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:54.726065 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d1eae633-2fce-4f7c-aebd-cf45f6fe03cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f27ce8a59f7a463e9106dd8f6ad0191b'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:54.727170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d1eae633-2fce-4f7c-aebd-cf45f6fe03cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f27ce8a59f7a463e9106dd8f6ad0191b'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:54.746522 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-d1eae633-2fce-4f7c-aebd-cf45f6fe03cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f27ce8a59f7a463e9106dd8f6ad0191b'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:54.775413 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1448/2885] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:16:54 2022] DELETE /identity/v3/users/f27ce8a59f7a463e9106dd8f6ad0191b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:16:55.106102 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:16:55.107605 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:16:55.129231 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267166db-a07f-4f2d-b3a9-68a1a68dfd3d admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:16:55.213675 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddefdf49-d3c3-47a5-809f-78ecbd5b9fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:16:55.214922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefdf49-d3c3-47a5-809f-78ecbd5b9fdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:16:55.214922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefdf49-d3c3-47a5-809f-78ecbd5b9fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:16:55.215203 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddefdf49-d3c3-47a5-809f-78ecbd5b9fdd admin admin] PATH_INFO: `/v3/projects/ef47c6c1b6b9478e80b5ec45add04f45` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:16:55.216581 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefdf49-d3c3-47a5-809f-78ecbd5b9fdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef47c6c1b6b9478e80b5ec45add04f45)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:16:55.219202 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddefdf49-d3c3-47a5-809f-78ecbd5b9fdd admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:16:55.299128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-ddefdf49-d3c3-47a5-809f-78ecbd5b9fdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef47c6c1b6b9478e80b5ec45add04f45', 'request_id': 'req-ddefdf49-d3c3-47a5-809f-78ecbd5b9fdd'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:55.310631 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-ddefdf49-d3c3-47a5-809f-78ecbd5b9fdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef47c6c1b6b9478e80b5ec45add04f45', 'request_id': 'req-ddefdf49-d3c3-47a5-809f-78ecbd5b9fdd'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:55.319538 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-ddefdf49-d3c3-47a5-809f-78ecbd5b9fdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef47c6c1b6b9478e80b5ec45add04f45', 'request_id': 'req-ddefdf49-d3c3-47a5-809f-78ecbd5b9fdd'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:55.331712 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-ddefdf49-d3c3-47a5-809f-78ecbd5b9fdd admin admin] The token cache is being invalidate because project ef47c6c1b6b9478e80b5ec45add04f45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:16:55.332043 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-ddefdf49-d3c3-47a5-809f-78ecbd5b9fdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:55.332658 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-ddefdf49-d3c3-47a5-809f-78ecbd5b9fdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:16:55.334414 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1438/2886] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:16:55 2022] DELETE /identity/v3/projects/ef47c6c1b6b9478e80b5ec45add04f45 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:17:01.762611 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f065e2-dd10-4631-9faf-1ca7285f64b3 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:01.763157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f065e2-dd10-4631-9faf-1ca7285f64b3 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:01.763157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f065e2-dd10-4631-9faf-1ca7285f64b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:01.783090 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-b6f065e2-dd10-4631-9faf-1ca7285f64b3 None None] MFA Rules not processed for user `d87960cdc11f47e2ac431484d3a56bd2`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:17:01.784484 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b6f065e2-dd10-4631-9faf-1ca7285f64b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:17:01.843224 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1449/2887] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:17:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:17:03.887857 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f1386-78eb-408e-8188-18c5ccc56523 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:03.888352 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f1386-78eb-408e-8188-18c5ccc56523 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:03.888439 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f1386-78eb-408e-8188-18c5ccc56523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:03.909885 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-353f1386-78eb-408e-8188-18c5ccc56523 None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:17:03.911246 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-353f1386-78eb-408e-8188-18c5ccc56523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:17:03.944959 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:17:03.973004 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1439/2888] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Sep 8 16:17:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:17:03.989777 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f065e2-dd10-4631-9faf-1ca7285f64b3 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:17:03.991790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-b6f065e2-dd10-4631-9faf-1ca7285f64b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:17:04.071157 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f065e2-dd10-4631-9faf-1ca7285f64b3 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:17:04.077259 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f22c5-d9a9-43d5-8063-5c97a2d198df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:17:04.077881 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f22c5-d9a9-43d5-8063-5c97a2d198df service swift] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:04.078323 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f22c5-d9a9-43d5-8063-5c97a2d198df service swift] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:04.078614 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f22c5-d9a9-43d5-8063-5c97a2d198df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:04.087839 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f22c5-d9a9-43d5-8063-5c97a2d198df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:17:04.089469 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f22c5-d9a9-43d5-8063-5c97a2d198df service swift] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:17:04.111990 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1450/2889] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:17:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:17:13.965340 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=8 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:17:14.095352 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:DELETE=5,keystone:SELECT=71,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:17:18.139204 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86c736-11ca-4016-97d1-caff2d642c9d None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:18.139789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86c736-11ca-4016-97d1-caff2d642c9d None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:18.139789 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed86c736-11ca-4016-97d1-caff2d642c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:18.161528 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-ed86c736-11ca-4016-97d1-caff2d642c9d None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:17:18.162729 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-ed86c736-11ca-4016-97d1-caff2d642c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:17:18.170480 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1440/2890] 10.222.0.27 () {58 vars in 1009 bytes} [Thu Sep 8 16:17:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:17:18.184750 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f22c5-d9a9-43d5-8063-5c97a2d198df service swift] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:17:18.196258 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f22c5-d9a9-43d5-8063-5c97a2d198df service swift] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:17:18.202154 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba299e-6cc2-4d4a-aa8c-5fd7db05ee44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:17:18.202781 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba299e-6cc2-4d4a-aa8c-5fd7db05ee44 service swift] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:18.203039 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba299e-6cc2-4d4a-aa8c-5fd7db05ee44 service swift] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:18.203299 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba299e-6cc2-4d4a-aa8c-5fd7db05ee44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:18.211479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba299e-6cc2-4d4a-aa8c-5fd7db05ee44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:17:18.213135 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba299e-6cc2-4d4a-aa8c-5fd7db05ee44 service swift] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:17:18.220986 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1451/2891] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:17:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:17:28.154418 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:17:28.217805 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=6 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:17:33.592919 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fdecc-1f18-457b-bb4c-9eba4f3ce0cc None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:33.593433 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fdecc-1f18-457b-bb4c-9eba4f3ce0cc None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:33.593532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3fdecc-1f18-457b-bb4c-9eba4f3ce0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:33.613320 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-4e3fdecc-1f18-457b-bb4c-9eba4f3ce0cc None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:17:33.614515 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4e3fdecc-1f18-457b-bb4c-9eba4f3ce0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:17:33.622368 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1441/2892] 10.222.0.27 () {58 vars in 967 bytes} [Thu Sep 8 16:17:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:17:33.634703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba299e-6cc2-4d4a-aa8c-5fd7db05ee44 service swift] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:17:33.645917 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba299e-6cc2-4d4a-aa8c-5fd7db05ee44 service swift] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:17:33.652508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f1eb3-2dcb-4d4f-81b2-9079607dd075 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:17:33.653103 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f1eb3-2dcb-4d4f-81b2-9079607dd075 service swift] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:33.653370 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f1eb3-2dcb-4d4f-81b2-9079607dd075 service swift] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:33.653633 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f1eb3-2dcb-4d4f-81b2-9079607dd075 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:33.662236 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39f1eb3-2dcb-4d4f-81b2-9079607dd075 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:17:33.663880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39f1eb3-2dcb-4d4f-81b2-9079607dd075 service swift] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:17:33.672012 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1452/2893] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:17:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:17:34.550127 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3fdecc-1f18-457b-bb4c-9eba4f3ce0cc None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:17:34.551828 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4e3fdecc-1f18-457b-bb4c-9eba4f3ce0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:17:34.617045 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3fdecc-1f18-457b-bb4c-9eba4f3ce0cc None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:17:34.621143 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19a8e5-5246-40fd-9e94-a6da5fd85298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:17:34.621750 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19a8e5-5246-40fd-9e94-a6da5fd85298 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:34.622023 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19a8e5-5246-40fd-9e94-a6da5fd85298 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:34.622302 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19a8e5-5246-40fd-9e94-a6da5fd85298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:34.624261 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-1a19a8e5-5246-40fd-9e94-a6da5fd85298 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:17:34.690050 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19a8e5-5246-40fd-9e94-a6da5fd85298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:17:34.691485 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19a8e5-5246-40fd-9e94-a6da5fd85298 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:17:34.701431 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1442/2894] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:17:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:17:35.672387 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f1eb3-2dcb-4d4f-81b2-9079607dd075 service swift] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:17:35.674692 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f39f1eb3-2dcb-4d4f-81b2-9079607dd075 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:17:35.748546 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f1eb3-2dcb-4d4f-81b2-9079607dd075 service swift] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:17:35.753030 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842330ae-15b4-452d-adfb-04163e0a193e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:17:35.753765 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842330ae-15b4-452d-adfb-04163e0a193e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:35.754042 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842330ae-15b4-452d-adfb-04163e0a193e admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:35.754311 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842330ae-15b4-452d-adfb-04163e0a193e admin admin] PATH_INFO: `/v3/users/b0d93e4e3e404b1799d3c64e86cb7049` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:35.755638 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842330ae-15b4-452d-adfb-04163e0a193e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0d93e4e3e404b1799d3c64e86cb7049)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:17:35.757757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842330ae-15b4-452d-adfb-04163e0a193e admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:17:35.812516 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-842330ae-15b4-452d-adfb-04163e0a193e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0d93e4e3e404b1799d3c64e86cb7049'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:17:35.820581 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-842330ae-15b4-452d-adfb-04163e0a193e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0d93e4e3e404b1799d3c64e86cb7049'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:17:35.821180 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-842330ae-15b4-452d-adfb-04163e0a193e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0d93e4e3e404b1799d3c64e86cb7049'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:17:35.821790 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-842330ae-15b4-452d-adfb-04163e0a193e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0d93e4e3e404b1799d3c64e86cb7049'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:17:35.834635 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-842330ae-15b4-452d-adfb-04163e0a193e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0d93e4e3e404b1799d3c64e86cb7049'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:17:35.846969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1453/2895] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:17:35 2022] DELETE /identity/v3/users/b0d93e4e3e404b1799d3c64e86cb7049 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:17:36.021551 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19a8e5-5246-40fd-9e94-a6da5fd85298 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:17:36.040608 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19a8e5-5246-40fd-9e94-a6da5fd85298 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:17:36.109704 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49e076b-5807-47ad-9a6b-f4605dc79bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:17:36.110260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e076b-5807-47ad-9a6b-f4605dc79bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:36.110532 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e076b-5807-47ad-9a6b-f4605dc79bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:36.110801 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49e076b-5807-47ad-9a6b-f4605dc79bd0 admin admin] PATH_INFO: `/v3/projects/2f722f1a8cc74bee978fe23ce9b6bf1c` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:36.111969 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49e076b-5807-47ad-9a6b-f4605dc79bd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f722f1a8cc74bee978fe23ce9b6bf1c)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:17:36.113890 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49e076b-5807-47ad-9a6b-f4605dc79bd0 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:17:36.159021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:17:36.162172 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f49e076b-5807-47ad-9a6b-f4605dc79bd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f722f1a8cc74bee978fe23ce9b6bf1c', 'request_id': 'req-f49e076b-5807-47ad-9a6b-f4605dc79bd0'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:17:36.175590 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f49e076b-5807-47ad-9a6b-f4605dc79bd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f722f1a8cc74bee978fe23ce9b6bf1c', 'request_id': 'req-f49e076b-5807-47ad-9a6b-f4605dc79bd0'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:17:36.186775 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f49e076b-5807-47ad-9a6b-f4605dc79bd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f722f1a8cc74bee978fe23ce9b6bf1c', 'request_id': 'req-f49e076b-5807-47ad-9a6b-f4605dc79bd0'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:17:36.198162 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f49e076b-5807-47ad-9a6b-f4605dc79bd0 admin admin] The token cache is being invalidate because project 2f722f1a8cc74bee978fe23ce9b6bf1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:17:36.198452 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f49e076b-5807-47ad-9a6b-f4605dc79bd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:17:36.199247 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f49e076b-5807-47ad-9a6b-f4605dc79bd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:17:36.200795 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1443/2896] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:17:36 2022] DELETE /identity/v3/projects/2f722f1a8cc74bee978fe23ce9b6bf1c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:17:36.205763 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac914a0e-df23-44a9-b0fc-32d1957725fc None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:36.206049 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac914a0e-df23-44a9-b0fc-32d1957725fc None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:36.206322 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac914a0e-df23-44a9-b0fc-32d1957725fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:36.226589 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-ac914a0e-df23-44a9-b0fc-32d1957725fc None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:17:36.227866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-ac914a0e-df23-44a9-b0fc-32d1957725fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:17:36.289290 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1454/2897] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:17:36.297522 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980be21d-36c0-4801-b88f-46a8ff37fdbc None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:36.297811 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980be21d-36c0-4801-b88f-46a8ff37fdbc None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:36.298077 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980be21d-36c0-4801-b88f-46a8ff37fdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:36.319398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-980be21d-36c0-4801-b88f-46a8ff37fdbc None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:17:36.320656 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-980be21d-36c0-4801-b88f-46a8ff37fdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:17:36.329874 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1444/2898] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:17:36.334236 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac914a0e-df23-44a9-b0fc-32d1957725fc None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:17:36.344206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac914a0e-df23-44a9-b0fc-32d1957725fc None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:17:36.349170 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fb0cd-bc77-42c3-83f1-4cd136aeebdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:17:36.349770 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fb0cd-bc77-42c3-83f1-4cd136aeebdb admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:36.350042 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fb0cd-bc77-42c3-83f1-4cd136aeebdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:36.350306 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fb0cd-bc77-42c3-83f1-4cd136aeebdb admin admin] PATH_INFO: `/v3/domains` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:36.350951 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fb0cd-bc77-42c3-83f1-4cd136aeebdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81526) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 16:17:36.351252 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fb0cd-bc77-42c3-83f1-4cd136aeebdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:17:36.352891 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fb0cd-bc77-42c3-83f1-4cd136aeebdb admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:17:36.354409 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:17:36.372324 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1455/2899] 10.222.0.27 () {58 vars in 1153 bytes} [Thu Sep 8 16:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 16:17:36.373118 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980be21d-36c0-4801-b88f-46a8ff37fdbc None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:17:36.382481 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980be21d-36c0-4801-b88f-46a8ff37fdbc None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:17:36.387639 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d9572-5b12-4234-ad24-8bd01a4371d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:17:36.388464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d9572-5b12-4234-ad24-8bd01a4371d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:36.388798 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d9572-5b12-4234-ad24-8bd01a4371d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:36.389119 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d9572-5b12-4234-ad24-8bd01a4371d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:36.389812 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d9572-5b12-4234-ad24-8bd01a4371d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:17:36.391807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d9572-5b12-4234-ad24-8bd01a4371d5 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:17:36.420294 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1445/2900] 10.222.0.27 () {60 vars in 1153 bytes} [Thu Sep 8 16:17:36 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:17:36.424521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fb0cd-bc77-42c3-83f1-4cd136aeebdb admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:17:36.434678 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fb0cd-bc77-42c3-83f1-4cd136aeebdb admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:17:36.493545 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdb675-f7de-4f90-9e6b-c4cd1d0a40d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:17:36.494164 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdb675-f7de-4f90-9e6b-c4cd1d0a40d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:36.494439 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdb675-f7de-4f90-9e6b-c4cd1d0a40d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:36.494703 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdb675-f7de-4f90-9e6b-c4cd1d0a40d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:36.495501 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafdb675-f7de-4f90-9e6b-c4cd1d0a40d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:17:36.497504 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafdb675-f7de-4f90-9e6b-c4cd1d0a40d7 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:17:36.519969 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1456/2901] 10.222.0.27 () {60 vars in 1144 bytes} [Thu Sep 8 16:17:36 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 16:17:36.524140 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d9572-5b12-4234-ad24-8bd01a4371d5 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:17:36.535158 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d9572-5b12-4234-ad24-8bd01a4371d5 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:17:36.540176 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1747f-556c-4ebf-a6c8-c19405ff385c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:17:36.540802 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1747f-556c-4ebf-a6c8-c19405ff385c admin admin] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:36.541082 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1747f-556c-4ebf-a6c8-c19405ff385c admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:36.541701 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1747f-556c-4ebf-a6c8-c19405ff385c admin admin] PATH_INFO: `/v3/roles` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:36.542488 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1747f-556c-4ebf-a6c8-c19405ff385c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:17:36.544145 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1747f-556c-4ebf-a6c8-c19405ff385c admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:17:36.559049 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1446/2902] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:17:36.560118 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdb675-f7de-4f90-9e6b-c4cd1d0a40d7 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:17:36.570041 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafdb675-f7de-4f90-9e6b-c4cd1d0a40d7 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:17:36.576185 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6850cb9-0c14-45f6-9c43-91fae62f7639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:17:36.576797 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6850cb9-0c14-45f6-9c43-91fae62f7639 admin admin] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:36.577145 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6850cb9-0c14-45f6-9c43-91fae62f7639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:36.577480 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6850cb9-0c14-45f6-9c43-91fae62f7639 admin admin] PATH_INFO: `/v3/roles` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:36.578203 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6850cb9-0c14-45f6-9c43-91fae62f7639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:17:36.580041 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6850cb9-0c14-45f6-9c43-91fae62f7639 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:17:36.595854 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1457/2903] 10.222.0.27 () {58 vars in 1122 bytes} [Thu Sep 8 16:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 16:17:36.599908 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1747f-556c-4ebf-a6c8-c19405ff385c admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:17:36.610403 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1747f-556c-4ebf-a6c8-c19405ff385c admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:17:36.615211 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b81c3-78f6-48bb-98d5-9657dc732515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:17:36.615925 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b81c3-78f6-48bb-98d5-9657dc732515 admin admin] REQUEST_METHOD: `PUT` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:36.616226 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b81c3-78f6-48bb-98d5-9657dc732515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:36.616612 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b81c3-78f6-48bb-98d5-9657dc732515 admin admin] PATH_INFO: `/v3/projects/40e4d03c01d54bbaa9bf5a76e2588cdd/users/f24ed20d44d044cfa43ccbdb35daab29/roles/abbd88ef58a544db8ee4261548778980` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:36.634032 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b81c3-78f6-48bb-98d5-9657dc732515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40e4d03c01d54bbaa9bf5a76e2588cdd, user_id=f24ed20d44d044cfa43ccbdb35daab29, role_id=abbd88ef58a544db8ee4261548778980)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:17:36.636781 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b81c3-78f6-48bb-98d5-9657dc732515 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:17:36.668890 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bed45-f22c-4186-b736-bcae17b90fd0 None None] REQUEST_METHOD: `POST` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:36.668890 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bed45-f22c-4186-b736-bcae17b90fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:36.669479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498bed45-f22c-4186-b736-bcae17b90fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:36.670406 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1447/2904] 10.222.0.27 () {60 vars in 1483 bytes} [Thu Sep 8 16:17:36 2022] PUT /identity/v3/projects/40e4d03c01d54bbaa9bf5a76e2588cdd/users/f24ed20d44d044cfa43ccbdb35daab29/roles/abbd88ef58a544db8ee4261548778980 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:17:36.711121 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.auth.core [None req-498bed45-f22c-4186-b736-bcae17b90fd0 None None] MFA Rules not processed for user `f24ed20d44d044cfa43ccbdb35daab29`. Rule list: `[]` (Enabled: `True`). {{(pid=81526) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:17:36.711961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-498bed45-f22c-4186-b736-bcae17b90fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:17:36.810778 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1458/2905] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:17:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:17:36.817398 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b81c3-78f6-48bb-98d5-9657dc732515 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:17:36.819924 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-7c6b81c3-78f6-48bb-98d5-9657dc732515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:17:36.905355 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b81c3-78f6-48bb-98d5-9657dc732515 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:17:36.909307 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4a85b-1ba1-4eec-b2a6-6b8dfecf69ab service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f24ee16c043244938901c4e850dea967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:17:36.910290 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4a85b-1ba1-4eec-b2a6-6b8dfecf69ab service glance] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:17:36.910567 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4a85b-1ba1-4eec-b2a6-6b8dfecf69ab service glance] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:17:36.910833 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4a85b-1ba1-4eec-b2a6-6b8dfecf69ab service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:17:36.922598 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4a85b-1ba1-4eec-b2a6-6b8dfecf69ab service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:17:36.924401 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f4a85b-1ba1-4eec-b2a6-6b8dfecf69ab service glance] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:17:36.946155 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1448/2906] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:17:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:17:46.798514 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=69,keystone:INSERT=3 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:17:46.932890 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:DELETE=4,keystone:SELECT=60,keystone:INSERT=3 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:18:07.578862 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bed45-f22c-4186-b736-bcae17b90fd0 None None] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:18:07.648200 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498bed45-f22c-4186-b736-bcae17b90fd0 None None] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:18:07.653252 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6cb55-190e-41f7-88ed-c95a04cd2d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:18:07.654344 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6cb55-190e-41f7-88ed-c95a04cd2d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:18:07.654344 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6cb55-190e-41f7-88ed-c95a04cd2d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:18:07.654531 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d6cb55-190e-41f7-88ed-c95a04cd2d9f admin admin] PATH_INFO: `/v3/users/f24ed20d44d044cfa43ccbdb35daab29` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:18:07.655701 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6cb55-190e-41f7-88ed-c95a04cd2d9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f24ed20d44d044cfa43ccbdb35daab29)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:18:07.657646 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d6cb55-190e-41f7-88ed-c95a04cd2d9f admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:18:07.709142 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-13d6cb55-190e-41f7-88ed-c95a04cd2d9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f24ed20d44d044cfa43ccbdb35daab29'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:07.718838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-13d6cb55-190e-41f7-88ed-c95a04cd2d9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f24ed20d44d044cfa43ccbdb35daab29'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:07.719455 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-13d6cb55-190e-41f7-88ed-c95a04cd2d9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f24ed20d44d044cfa43ccbdb35daab29'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:07.720162 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-13d6cb55-190e-41f7-88ed-c95a04cd2d9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f24ed20d44d044cfa43ccbdb35daab29'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:07.732757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-13d6cb55-190e-41f7-88ed-c95a04cd2d9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f24ed20d44d044cfa43ccbdb35daab29'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:07.746737 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1459/2907] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:18:07 2022] DELETE /identity/v3/users/f24ed20d44d044cfa43ccbdb35daab29 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:18:07.752333 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612b54f-ffaf-4c72-957a-d883c003b3e8 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:18:07.752865 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612b54f-ffaf-4c72-957a-d883c003b3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:18:07.752865 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f612b54f-ffaf-4c72-957a-d883c003b3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:18:07.775812 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-f612b54f-ffaf-4c72-957a-d883c003b3e8 None None] MFA Rules not processed for user `02fc092547d04cbf984e632c0755920a`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:18:07.777419 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f612b54f-ffaf-4c72-957a-d883c003b3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:18:07.841209 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1449/2908] 10.222.0.27 () {58 vars in 958 bytes} [Thu Sep 8 16:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:18:07.851410 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6cb55-190e-41f7-88ed-c95a04cd2d9f admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:18:07.853089 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-13d6cb55-190e-41f7-88ed-c95a04cd2d9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:18:07.915631 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d6cb55-190e-41f7-88ed-c95a04cd2d9f admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:18:07.919947 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9b96e-8bb9-4bb7-8adc-cd0eff0c46ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:18:07.920552 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9b96e-8bb9-4bb7-8adc-cd0eff0c46ea service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:18:07.920858 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9b96e-8bb9-4bb7-8adc-cd0eff0c46ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:18:07.921126 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de9b96e-8bb9-4bb7-8adc-cd0eff0c46ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:18:07.931356 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9b96e-8bb9-4bb7-8adc-cd0eff0c46ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:18:07.933201 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de9b96e-8bb9-4bb7-8adc-cd0eff0c46ea service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:18:07.948316 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1460/2909] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:18:08.219745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612b54f-ffaf-4c72-957a-d883c003b3e8 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:18:08.221007 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f612b54f-ffaf-4c72-957a-d883c003b3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:18:08.234490 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f612b54f-ffaf-4c72-957a-d883c003b3e8 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:18:08.239629 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c376c98-c492-499b-8a48-86d8a7f29cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:18:08.240365 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c376c98-c492-499b-8a48-86d8a7f29cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:18:08.241571 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c376c98-c492-499b-8a48-86d8a7f29cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:18:08.241853 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c376c98-c492-499b-8a48-86d8a7f29cdb admin admin] PATH_INFO: `/v3/projects/40e4d03c01d54bbaa9bf5a76e2588cdd` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:18:08.243381 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c376c98-c492-499b-8a48-86d8a7f29cdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=40e4d03c01d54bbaa9bf5a76e2588cdd)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:18:08.245600 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c376c98-c492-499b-8a48-86d8a7f29cdb admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:18:08.296850 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2c376c98-c492-499b-8a48-86d8a7f29cdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40e4d03c01d54bbaa9bf5a76e2588cdd', 'request_id': 'req-2c376c98-c492-499b-8a48-86d8a7f29cdb'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:08.304714 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2c376c98-c492-499b-8a48-86d8a7f29cdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40e4d03c01d54bbaa9bf5a76e2588cdd', 'request_id': 'req-2c376c98-c492-499b-8a48-86d8a7f29cdb'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:08.311389 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2c376c98-c492-499b-8a48-86d8a7f29cdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40e4d03c01d54bbaa9bf5a76e2588cdd', 'request_id': 'req-2c376c98-c492-499b-8a48-86d8a7f29cdb'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:08.324587 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2c376c98-c492-499b-8a48-86d8a7f29cdb admin admin] The token cache is being invalidate because project 40e4d03c01d54bbaa9bf5a76e2588cdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:18:08.325022 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2c376c98-c492-499b-8a48-86d8a7f29cdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:08.325818 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-2c376c98-c492-499b-8a48-86d8a7f29cdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:08.327387 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1450/2910] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:18:08 2022] DELETE /identity/v3/projects/40e4d03c01d54bbaa9bf5a76e2588cdd => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:18:10.912661 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9b96e-8bb9-4bb7-8adc-cd0eff0c46ea service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:18:10.921958 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de9b96e-8bb9-4bb7-8adc-cd0eff0c46ea service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:18:10.941535 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:18:10.975392 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824a7b4-5af9-41b6-ae98-ef7db8f2a5d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:18:10.976059 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824a7b4-5af9-41b6-ae98-ef7db8f2a5d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:18:10.976451 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824a7b4-5af9-41b6-ae98-ef7db8f2a5d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:18:10.976722 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824a7b4-5af9-41b6-ae98-ef7db8f2a5d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:18:10.978491 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e824a7b4-5af9-41b6-ae98-ef7db8f2a5d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:18:11.060897 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824a7b4-5af9-41b6-ae98-ef7db8f2a5d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:18:11.062403 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824a7b4-5af9-41b6-ae98-ef7db8f2a5d5 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:18:11.083166 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1461/2911] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:18:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:18:12.349570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c376c98-c492-499b-8a48-86d8a7f29cdb admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:18:12.351002 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-2c376c98-c492-499b-8a48-86d8a7f29cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:18:12.365922 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c376c98-c492-499b-8a48-86d8a7f29cdb admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:18:12.369915 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4da47-ed07-4046-9a22-ddf4d9f5bbfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:18:12.370516 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4da47-ed07-4046-9a22-ddf4d9f5bbfa service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:18:12.370787 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4da47-ed07-4046-9a22-ddf4d9f5bbfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:18:12.371048 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde4da47-ed07-4046-9a22-ddf4d9f5bbfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:18:12.373790 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-bde4da47-ed07-4046-9a22-ddf4d9f5bbfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:18:12.390382 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4da47-ed07-4046-9a22-ddf4d9f5bbfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:18:12.391848 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde4da47-ed07-4046-9a22-ddf4d9f5bbfa service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:18:12.405609 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1451/2912] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:18:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:18:21.080763 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=53 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:18:22.397940 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:18:24.743155 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824a7b4-5af9-41b6-ae98-ef7db8f2a5d5 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:18:24.752737 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824a7b4-5af9-41b6-ae98-ef7db8f2a5d5 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:18:24.757237 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f626f94-bde6-465c-8af4-1ecc1cd343f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:18:24.757932 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f626f94-bde6-465c-8af4-1ecc1cd343f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:18:24.758427 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f626f94-bde6-465c-8af4-1ecc1cd343f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:18:24.758902 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f626f94-bde6-465c-8af4-1ecc1cd343f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:18:24.760963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3f626f94-bde6-465c-8af4-1ecc1cd343f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:18:24.771584 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f626f94-bde6-465c-8af4-1ecc1cd343f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:18:24.773199 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f626f94-bde6-465c-8af4-1ecc1cd343f8 service neutron] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:18:24.783213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1462/2913] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:18:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:18:30.527871 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4da47-ed07-4046-9a22-ddf4d9f5bbfa service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:18:30.529489 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-bde4da47-ed07-4046-9a22-ddf4d9f5bbfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:18:30.540861 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde4da47-ed07-4046-9a22-ddf4d9f5bbfa service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:18:30.545330 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc831e52-598b-4e8d-9835-14558b150284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aae65f9ec394f97aeafad43932dc49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:18:30.545934 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc831e52-598b-4e8d-9835-14558b150284 service neutron] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:18:30.546207 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc831e52-598b-4e8d-9835-14558b150284 service neutron] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:18:30.546476 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc831e52-598b-4e8d-9835-14558b150284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:18:30.548329 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-dc831e52-598b-4e8d-9835-14558b150284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:18:30.613217 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc831e52-598b-4e8d-9835-14558b150284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:18:30.614668 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc831e52-598b-4e8d-9835-14558b150284 service neutron] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:18:30.624906 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1452/2914] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:18:34.778274 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=6 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:18:36.717215 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f626f94-bde6-465c-8af4-1ecc1cd343f8 service neutron] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:18:36.718772 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-3f626f94-bde6-465c-8af4-1ecc1cd343f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:18:36.739335 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f626f94-bde6-465c-8af4-1ecc1cd343f8 service neutron] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:18:36.744125 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6c3df-6223-4ea8-96ab-c817909dca21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:18:36.744906 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6c3df-6223-4ea8-96ab-c817909dca21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:18:36.745193 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6c3df-6223-4ea8-96ab-c817909dca21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:18:36.745447 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6c3df-6223-4ea8-96ab-c817909dca21 admin admin] PATH_INFO: `/v3/users/77aeaed3d55945e49b0eb6e3c0519294` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:18:36.747192 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6c3df-6223-4ea8-96ab-c817909dca21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77aeaed3d55945e49b0eb6e3c0519294)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:18:36.751488 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6c3df-6223-4ea8-96ab-c817909dca21 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:18:36.802634 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e6c6c3df-6223-4ea8-96ab-c817909dca21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77aeaed3d55945e49b0eb6e3c0519294'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:36.811775 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e6c6c3df-6223-4ea8-96ab-c817909dca21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77aeaed3d55945e49b0eb6e3c0519294'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:36.812422 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e6c6c3df-6223-4ea8-96ab-c817909dca21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77aeaed3d55945e49b0eb6e3c0519294'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:36.813028 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e6c6c3df-6223-4ea8-96ab-c817909dca21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77aeaed3d55945e49b0eb6e3c0519294'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:36.824880 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-e6c6c3df-6223-4ea8-96ab-c817909dca21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77aeaed3d55945e49b0eb6e3c0519294'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:36.837060 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1463/2915] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:18:36 2022] DELETE /identity/v3/users/77aeaed3d55945e49b0eb6e3c0519294 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:18:37.022801 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc831e52-598b-4e8d-9835-14558b150284 service neutron] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:18:37.033411 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc831e52-598b-4e8d-9835-14558b150284 service neutron] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:18:37.093140 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537529c-8b5e-4db8-9524-3cd6a19c51f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:18:37.093815 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537529c-8b5e-4db8-9524-3cd6a19c51f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:18:37.094092 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537529c-8b5e-4db8-9524-3cd6a19c51f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:18:37.094419 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4537529c-8b5e-4db8-9524-3cd6a19c51f4 admin admin] PATH_INFO: `/v3/projects/53275b4dc845465d8b0bfb50eacd14f6` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:18:37.095664 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537529c-8b5e-4db8-9524-3cd6a19c51f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53275b4dc845465d8b0bfb50eacd14f6)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:18:37.097570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4537529c-8b5e-4db8-9524-3cd6a19c51f4 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:18:37.144769 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-4537529c-8b5e-4db8-9524-3cd6a19c51f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53275b4dc845465d8b0bfb50eacd14f6', 'request_id': 'req-4537529c-8b5e-4db8-9524-3cd6a19c51f4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:37.153260 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-4537529c-8b5e-4db8-9524-3cd6a19c51f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53275b4dc845465d8b0bfb50eacd14f6', 'request_id': 'req-4537529c-8b5e-4db8-9524-3cd6a19c51f4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:37.158953 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-4537529c-8b5e-4db8-9524-3cd6a19c51f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53275b4dc845465d8b0bfb50eacd14f6', 'request_id': 'req-4537529c-8b5e-4db8-9524-3cd6a19c51f4'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:37.169880 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-4537529c-8b5e-4db8-9524-3cd6a19c51f4 admin admin] The token cache is being invalidate because project 53275b4dc845465d8b0bfb50eacd14f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:18:37.170169 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-4537529c-8b5e-4db8-9524-3cd6a19c51f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:37.170815 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-4537529c-8b5e-4db8-9524-3cd6a19c51f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:37.172374 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1453/2916] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:18:37 2022] DELETE /identity/v3/projects/53275b4dc845465d8b0bfb50eacd14f6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:18:37.176697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6c3df-6223-4ea8-96ab-c817909dca21 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:18:37.178174 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e6c6c3df-6223-4ea8-96ab-c817909dca21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:18:37.191949 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6c3df-6223-4ea8-96ab-c817909dca21 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:18:37.250630 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc1f4c-fe57-4eba-aabf-6fe0080e822d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:18:37.251265 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc1f4c-fe57-4eba-aabf-6fe0080e822d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:18:37.251570 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc1f4c-fe57-4eba-aabf-6fe0080e822d admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:18:37.251832 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc1f4c-fe57-4eba-aabf-6fe0080e822d admin admin] PATH_INFO: `/v3/users/02cddad0448d45bb9458d5cc7cdfbc5a` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:18:37.253091 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc1f4c-fe57-4eba-aabf-6fe0080e822d admin admin] RBAC: Authorizing `identity:delete_user(user_id=02cddad0448d45bb9458d5cc7cdfbc5a)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:18:37.255018 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc1f4c-fe57-4eba-aabf-6fe0080e822d admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:18:37.305521 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-05bc1f4c-fe57-4eba-aabf-6fe0080e822d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02cddad0448d45bb9458d5cc7cdfbc5a'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:37.315010 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-05bc1f4c-fe57-4eba-aabf-6fe0080e822d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02cddad0448d45bb9458d5cc7cdfbc5a'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:37.315757 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-05bc1f4c-fe57-4eba-aabf-6fe0080e822d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02cddad0448d45bb9458d5cc7cdfbc5a'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:37.316425 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-05bc1f4c-fe57-4eba-aabf-6fe0080e822d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02cddad0448d45bb9458d5cc7cdfbc5a'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:37.328435 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-05bc1f4c-fe57-4eba-aabf-6fe0080e822d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02cddad0448d45bb9458d5cc7cdfbc5a'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:37.330500 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:18:37.342963 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1464/2917] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:18:37 2022] DELETE /identity/v3/users/02cddad0448d45bb9458d5cc7cdfbc5a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:18:37.527570 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537529c-8b5e-4db8-9524-3cd6a19c51f4 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:18:37.529041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-4537529c-8b5e-4db8-9524-3cd6a19c51f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:18:37.543025 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4537529c-8b5e-4db8-9524-3cd6a19c51f4 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:18:37.564475 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:18:37.602322 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b096c-05ed-45d7-91e2-5a25d8a7d851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:18:37.602997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b096c-05ed-45d7-91e2-5a25d8a7d851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:18:37.603273 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b096c-05ed-45d7-91e2-5a25d8a7d851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:18:37.603580 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b096c-05ed-45d7-91e2-5a25d8a7d851 admin admin] PATH_INFO: `/v3/projects/19d1c7a3b9154b0aad0c4286f3199772` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:18:37.604711 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b096c-05ed-45d7-91e2-5a25d8a7d851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19d1c7a3b9154b0aad0c4286f3199772)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:18:37.606622 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b096c-05ed-45d7-91e2-5a25d8a7d851 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:18:37.655141 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1c9b096c-05ed-45d7-91e2-5a25d8a7d851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19d1c7a3b9154b0aad0c4286f3199772', 'request_id': 'req-1c9b096c-05ed-45d7-91e2-5a25d8a7d851'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:37.663412 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1c9b096c-05ed-45d7-91e2-5a25d8a7d851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19d1c7a3b9154b0aad0c4286f3199772', 'request_id': 'req-1c9b096c-05ed-45d7-91e2-5a25d8a7d851'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:37.669427 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1c9b096c-05ed-45d7-91e2-5a25d8a7d851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19d1c7a3b9154b0aad0c4286f3199772', 'request_id': 'req-1c9b096c-05ed-45d7-91e2-5a25d8a7d851'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:37.681770 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1c9b096c-05ed-45d7-91e2-5a25d8a7d851 admin admin] The token cache is being invalidate because project 19d1c7a3b9154b0aad0c4286f3199772 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:18:37.682106 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1c9b096c-05ed-45d7-91e2-5a25d8a7d851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:37.682652 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-1c9b096c-05ed-45d7-91e2-5a25d8a7d851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:18:37.684353 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1454/2918] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:18:37 2022] DELETE /identity/v3/projects/19d1c7a3b9154b0aad0c4286f3199772 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:18:47.342477 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=4 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:18:47.678676 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:18:56.507819 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc1f4c-fe57-4eba-aabf-6fe0080e822d admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:18:56.510165 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-05bc1f4c-fe57-4eba-aabf-6fe0080e822d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:18:56.599696 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc1f4c-fe57-4eba-aabf-6fe0080e822d admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:18:56.679071 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92905341-a861-4a95-91b1-4752031147e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e9209e77ce44b128e25a8ddcf78f8b5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:18:56.680883 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92905341-a861-4a95-91b1-4752031147e2 service nova] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:18:56.681669 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92905341-a861-4a95-91b1-4752031147e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:18:56.681952 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92905341-a861-4a95-91b1-4752031147e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:18:56.683915 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-92905341-a861-4a95-91b1-4752031147e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:18:56.696866 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92905341-a861-4a95-91b1-4752031147e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:18:56.698809 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92905341-a861-4a95-91b1-4752031147e2 service nova] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:18:56.712160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1465/2919] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:18:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:19:04.488870 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e59c0-29eb-4926-9ec6-9a2f74821bc9 None None] REQUEST_METHOD: `POST` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:19:04.489384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e59c0-29eb-4926-9ec6-9a2f74821bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:19:04.489384 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e59c0-29eb-4926-9ec6-9a2f74821bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:19:04.539128 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.auth.core [None req-fd0e59c0-29eb-4926-9ec6-9a2f74821bc9 None None] MFA Rules not processed for user `b4324e4c04e343c98d9bb216c542fcba`. Rule list: `[]` (Enabled: `True`). {{(pid=81527) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 16:19:04.540361 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-fd0e59c0-29eb-4926-9ec6-9a2f74821bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:19:04.615603 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1455/2920] 10.222.0.27 () {58 vars in 967 bytes} [Thu Sep 8 16:19:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 16:19:04.665417 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92905341-a861-4a95-91b1-4752031147e2 service nova] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:19:04.666914 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-92905341-a861-4a95-91b1-4752031147e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:19:04.761120 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92905341-a861-4a95-91b1-4752031147e2 service nova] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:19:04.766338 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558ec75-5470-45f8-a501-49379de04ac6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'de54687547a14ab89cc41a1fabaab400', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:19:04.766939 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558ec75-5470-45f8-a501-49379de04ac6 service swift] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:19:04.767213 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558ec75-5470-45f8-a501-49379de04ac6 service swift] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:19:04.767508 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558ec75-5470-45f8-a501-49379de04ac6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:19:04.772478 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=100 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:19:04.776479 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558ec75-5470-45f8-a501-49379de04ac6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:19:04.778329 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558ec75-5470-45f8-a501-49379de04ac6 service swift] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:19:04.786838 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1466/2921] 10.222.0.27 () {60 vars in 1384 bytes} [Thu Sep 8 16:19:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 16:19:07.949592 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e59c0-29eb-4926-9ec6-9a2f74821bc9 None None] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:19:07.951266 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-fd0e59c0-29eb-4926-9ec6-9a2f74821bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:19:08.054884 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e59c0-29eb-4926-9ec6-9a2f74821bc9 None None] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:19:08.065060 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0471a-eaca-478f-846d-262bc60a2e15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd28bb31b55e45668212c9ab69ed3ac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:19:08.065782 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0471a-eaca-478f-846d-262bc60a2e15 service placement] REQUEST_METHOD: `GET` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:19:08.066120 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0471a-eaca-478f-846d-262bc60a2e15 service placement] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:19:08.066450 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0471a-eaca-478f-846d-262bc60a2e15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:19:08.068450 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-8cf0471a-eaca-478f-846d-262bc60a2e15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:19:08.082041 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf0471a-eaca-478f-846d-262bc60a2e15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:19:08.083787 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf0471a-eaca-478f-846d-262bc60a2e15 service placement] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:19:08.095528 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1456/2922] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:19:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:19:11.515697 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558ec75-5470-45f8-a501-49379de04ac6 service swift] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:19:11.517184 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-7558ec75-5470-45f8-a501-49379de04ac6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:19:11.593760 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558ec75-5470-45f8-a501-49379de04ac6 service swift] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:19:11.598941 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94763930-8bcf-4a30-a74e-0c1873f33df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:19:11.599627 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94763930-8bcf-4a30-a74e-0c1873f33df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:19:11.599886 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94763930-8bcf-4a30-a74e-0c1873f33df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:19:11.600149 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94763930-8bcf-4a30-a74e-0c1873f33df7 admin admin] PATH_INFO: `/v3/users/d87960cdc11f47e2ac431484d3a56bd2` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:19:11.601378 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94763930-8bcf-4a30-a74e-0c1873f33df7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d87960cdc11f47e2ac431484d3a56bd2)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:19:11.603326 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94763930-8bcf-4a30-a74e-0c1873f33df7 admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:19:11.653507 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-94763930-8bcf-4a30-a74e-0c1873f33df7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd87960cdc11f47e2ac431484d3a56bd2'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:11.663738 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-94763930-8bcf-4a30-a74e-0c1873f33df7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd87960cdc11f47e2ac431484d3a56bd2'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:11.664418 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-94763930-8bcf-4a30-a74e-0c1873f33df7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd87960cdc11f47e2ac431484d3a56bd2'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:11.665035 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-94763930-8bcf-4a30-a74e-0c1873f33df7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd87960cdc11f47e2ac431484d3a56bd2'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:11.676752 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-94763930-8bcf-4a30-a74e-0c1873f33df7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd87960cdc11f47e2ac431484d3a56bd2'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:11.690875 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1467/2923] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:19:11 2022] DELETE /identity/v3/users/d87960cdc11f47e2ac431484d3a56bd2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:19:11.873705 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0471a-eaca-478f-846d-262bc60a2e15 service placement] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:19:11.885069 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0471a-eaca-478f-846d-262bc60a2e15 service placement] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:19:11.943691 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9acd20c-22d0-40fe-8f0b-403a23f451a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:19:11.944391 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acd20c-22d0-40fe-8f0b-403a23f451a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:19:11.944671 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acd20c-22d0-40fe-8f0b-403a23f451a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:19:11.944949 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9acd20c-22d0-40fe-8f0b-403a23f451a9 admin admin] PATH_INFO: `/v3/projects/223d0d2bc7a54f5b9ccc4b4414604d87` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:19:11.946191 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9acd20c-22d0-40fe-8f0b-403a23f451a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=223d0d2bc7a54f5b9ccc4b4414604d87)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:19:11.948228 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9acd20c-22d0-40fe-8f0b-403a23f451a9 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:19:11.950583 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=100 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:19:12.008807 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f9acd20c-22d0-40fe-8f0b-403a23f451a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '223d0d2bc7a54f5b9ccc4b4414604d87', 'request_id': 'req-f9acd20c-22d0-40fe-8f0b-403a23f451a9'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:12.021745 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f9acd20c-22d0-40fe-8f0b-403a23f451a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '223d0d2bc7a54f5b9ccc4b4414604d87', 'request_id': 'req-f9acd20c-22d0-40fe-8f0b-403a23f451a9'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:12.031659 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f9acd20c-22d0-40fe-8f0b-403a23f451a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '223d0d2bc7a54f5b9ccc4b4414604d87', 'request_id': 'req-f9acd20c-22d0-40fe-8f0b-403a23f451a9'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:12.043725 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f9acd20c-22d0-40fe-8f0b-403a23f451a9 admin admin] The token cache is being invalidate because project 223d0d2bc7a54f5b9ccc4b4414604d87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:19:12.044011 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f9acd20c-22d0-40fe-8f0b-403a23f451a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:12.044661 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-f9acd20c-22d0-40fe-8f0b-403a23f451a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:12.046190 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1457/2924] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:19:11 2022] DELETE /identity/v3/projects/223d0d2bc7a54f5b9ccc4b4414604d87 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:19:12.051261 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94763930-8bcf-4a30-a74e-0c1873f33df7 admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:19:12.052741 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-94763930-8bcf-4a30-a74e-0c1873f33df7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:19:12.066551 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94763930-8bcf-4a30-a74e-0c1873f33df7 admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:19:12.125867 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eab640-b958-4efe-9720-bbb88efa0c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:19:12.126850 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eab640-b958-4efe-9720-bbb88efa0c6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:19:12.127194 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eab640-b958-4efe-9720-bbb88efa0c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:19:12.127563 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eab640-b958-4efe-9720-bbb88efa0c6b admin admin] PATH_INFO: `/v3/users/c1c6b313f3a1481a92ad373f25504e55` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:19:12.129349 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eab640-b958-4efe-9720-bbb88efa0c6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c6b313f3a1481a92ad373f25504e55)` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:19:12.132436 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eab640-b958-4efe-9720-bbb88efa0c6b admin admin] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:19:12.148576 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:19:12.183723 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f1eab640-b958-4efe-9720-bbb88efa0c6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c6b313f3a1481a92ad373f25504e55'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:12.191961 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f1eab640-b958-4efe-9720-bbb88efa0c6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c6b313f3a1481a92ad373f25504e55'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:12.192562 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f1eab640-b958-4efe-9720-bbb88efa0c6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c6b313f3a1481a92ad373f25504e55'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:12.193160 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f1eab640-b958-4efe-9720-bbb88efa0c6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c6b313f3a1481a92ad373f25504e55'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:12.205283 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.notifications [None req-f1eab640-b958-4efe-9720-bbb88efa0c6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c6b313f3a1481a92ad373f25504e55'} {{(pid=81526) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:12.217297 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1468/2925] 10.222.0.27 () {58 vars in 1224 bytes} [Thu Sep 8 16:19:12 2022] DELETE /identity/v3/users/c1c6b313f3a1481a92ad373f25504e55 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:19:12.409244 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9acd20c-22d0-40fe-8f0b-403a23f451a9 admin admin] Authenticating user token {{(pid=81527) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:19:12.410720 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.fernet_utils [None req-f9acd20c-22d0-40fe-8f0b-403a23f451a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81527) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:19:12.425140 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9acd20c-22d0-40fe-8f0b-403a23f451a9 admin admin] Validating token access rules against request {{(pid=81527) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:19:12.492051 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26e20c-0e55-40d0-8760-f76323043727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '02fc092547d04cbf984e632c0755920a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e21b84c9f53a40aa98d4a318db545714', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81527) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:19:12.492686 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26e20c-0e55-40d0-8760-f76323043727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:19:12.492969 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26e20c-0e55-40d0-8760-f76323043727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:19:12.493241 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26e20c-0e55-40d0-8760-f76323043727 admin admin] PATH_INFO: `/v3/projects/32e079fde1ca49f3a2bbcf5da1d35d6d` {{(pid=81527) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:19:12.494373 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26e20c-0e55-40d0-8760-f76323043727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32e079fde1ca49f3a2bbcf5da1d35d6d)` {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:19:12.496472 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26e20c-0e55-40d0-8760-f76323043727 admin admin] RBAC: Authorization granted {{(pid=81527) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:19:12.545927 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6a26e20c-0e55-40d0-8760-f76323043727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32e079fde1ca49f3a2bbcf5da1d35d6d', 'request_id': 'req-6a26e20c-0e55-40d0-8760-f76323043727'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:12.554021 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6a26e20c-0e55-40d0-8760-f76323043727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32e079fde1ca49f3a2bbcf5da1d35d6d', 'request_id': 'req-6a26e20c-0e55-40d0-8760-f76323043727'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:12.560464 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6a26e20c-0e55-40d0-8760-f76323043727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32e079fde1ca49f3a2bbcf5da1d35d6d', 'request_id': 'req-6a26e20c-0e55-40d0-8760-f76323043727'} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:12.571997 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6a26e20c-0e55-40d0-8760-f76323043727 admin admin] The token cache is being invalidate because project 32e079fde1ca49f3a2bbcf5da1d35d6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81527) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 16:19:12.572651 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6a26e20c-0e55-40d0-8760-f76323043727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:12.573461 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG keystone.notifications [None req-6a26e20c-0e55-40d0-8760-f76323043727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81527) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 16:19:12.575049 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: [pid: 81527|app: 0|req: 1458/2926] 10.222.0.27 () {58 vars in 1233 bytes} [Thu Sep 8 16:19:12 2022] DELETE /identity/v3/projects/32e079fde1ca49f3a2bbcf5da1d35d6d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 16:19:22.214003 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=21,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:19:22.568935 dv-cinder-856259-2-iscsi devstack@keystone.service[81527]: DEBUG dbcounter [-] [81527] Writing DB stats keystone:SELECT=74,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81527) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 16:21:10.759618 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eab640-b958-4efe-9720-bbb88efa0c6b admin admin] Authenticating user token {{(pid=81526) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 16:21:10.761030 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-f1eab640-b958-4efe-9720-bbb88efa0c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:21:10.835487 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eab640-b958-4efe-9720-bbb88efa0c6b admin admin] Validating token access rules against request {{(pid=81526) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 16:21:10.898013 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e8a93-f2ae-46ec-bc97-8eddf0d992a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd28bb31b55e45668212c9ab69ed3ac3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a825a0b1238d4382995da440944f453b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81526) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 16:21:10.898625 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e8a93-f2ae-46ec-bc97-8eddf0d992a7 service placement] REQUEST_METHOD: `GET` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 16:21:10.898901 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e8a93-f2ae-46ec-bc97-8eddf0d992a7 service placement] SCRIPT_NAME: `/identity` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 16:21:10.899182 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e8a93-f2ae-46ec-bc97-8eddf0d992a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81526) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 16:21:10.901198 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.fernet_utils [None req-e39e8a93-f2ae-46ec-bc97-8eddf0d992a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81526) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 16:21:10.912621 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e8a93-f2ae-46ec-bc97-8eddf0d992a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 16:21:10.914206 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e8a93-f2ae-46ec-bc97-8eddf0d992a7 service placement] RBAC: Authorization granted {{(pid=81526) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 16:21:10.924201 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: [pid: 81526|app: 0|req: 1469/2927] 10.222.0.27 () {60 vars in 1365 bytes} [Thu Sep 8 16:21:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 16:21:20.920098 dv-cinder-856259-2-iscsi devstack@keystone.service[81526]: DEBUG dbcounter [-] [81526] Writing DB stats keystone:SELECT=58 {{(pid=81526) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}